Tips

This document just covers some helpful tips, understandings, and tooling I built to help make updating and mapping packets a little easier.

If you haven't already read the mapping guide check it out. The mapping guide contains information which will help make some of these tips make a little more sense.

Creating an Updater

My general strategy for creating an automated updater came in three steps:

Fingerprinting the client
Extracting key data (Extractors)
Writing to a simple JSON format to be ingested by my API and other tooling

When I say "fingerprinting the client" I simply mean creating a set of classes which can uniquely identify core parts of the client (ClassNode and MethodNode) which contain information I want to extract (i.e. packet information or general reflection data).

For example,

java

public interface Fingerprinter {
    AnalysisTarget target();

    default Set<AnalysisTarget> dependencies() {
        return Collections.emptySet();
    }

    Fingerprint search(AnalysisSession session);
}

This interface lets me define a target I am searching for like: DO_ACTION, EVENT_MOUSE_CLICK_SENDER, MOVEMENT_PACKET_SENDER etc..., a set of dependencies on other fingerprints which may be required to find the target, and a method which will search for the target. This builds a tree of fingerprints which can each be executed to find key classes and methods holding information I want to extract.

Extractors do exactly what they sound like, they extract information from the client using what was found in the fingerprinting process. For example,

java

public interface MetadataExtractor {

    default Set<AnalysisTarget> requiredFingerprints() {
        return Collections.emptySet();
    }

    default Set<MetadataKey> requiredMetadata() {
        return Collections.emptySet();
    }

    Set<MetadataKey> providedMetadata();

    MetadataFragment extract(AnalysisSession session);
}

This interfaces lets me define a set of extractors which use the classes and methods found in the fingerprinting process to extract information. They function similarly to fingerprints in that they can have dependencies on other extractors or fingerprints to find what they need.

This lets the core part of my client updater look like this:

java

AnnotationRemover.stripNamedAnnotations(inputPath, outputPath);
Map<String, ClassNode> classes = JarUtils.loadClasses(outputPath);
AnalysisSession session = new AnalysisSession(classes);

fingerprintManager.runAll(session);
deobfuscationService.deobfuscate(session);
extractorManager.runAll(session);
JarUtils.writeJar(outputPath, classes, outputPath);
// ...
Map<String, PacketDefinition> mappedPackets = mapper.run(session);

try (Writer writer = new FileWriter(output.toString())) {
    gson.toJson(new MappedPackets(clientVersion, session.metadata(), mappedPackets), writer);
}

Bytecode Instruction Shuffling

The first hurdle in building an automated updater is dealing with bytecode jump instructions. RuneLite/OSRS mixes up the bytecode of their methods and stitches them together with jump instructions, which makes the bytecode itself hard to read & follow in a logical order.

Decompilers like Fernflower are great for decompiling bytecode and will automatically re-organize the bytecode instructions into a defined order to make it easier to read. However, sometimes it's useful to see the actual bytecode for yourself to help identify patterns for parsing information. In these scenarios, it's helpful to have a tool to re-structure the bytecode instructions into a logical order. I wrote a BytecodeLinearizer class which can be used to do this as part of my tooling and its helped tremendously in understanding the actual bytecode instructions that are being iterated over with ASM.

Write method ordering

Every revision the OSRS client shuffles the ordering of their write methods. Failing to write data into the packet in the right order will produce malformed packets which will quickly fail. For example, in the OPOBJT packet the writes may currently be written in the order:

(slot, objectId, worldPointY, widgetId, worldPointX, itemId, ctrlDown)

However, the next revision the ordering may be:

(objectId, worldPointY, itemId, objectId, slot, widgetId, worldPointX)

Method ordering is challenging to automatically determine through bytecode analysis alone as you have to trace the variables being passed into the method through the client. For some methods where you know the order of arguments to the method and they are passed directly into the write method, the ordering can be determined in a fairly straightforward manner.

For example,

java

public static void dw(int var0, int var1, int var2) {
    // ...
    jg var82 = hk.ah(iq.dm, client.ay.az, (byte)89);
    wg var104 = var82.ai;
    
    // Since arg order doesn't change if we know var1 is playerIndex
    // then we know that playerIndex will be the first write for this packet
    var104.dw(var1, (byte)-16);
    var82.ai.eu(var18, 499124101);
    client.ay.ae(var82, (byte)-62);
}

Building Effective Tooling

If you really want to automate the packet mapping process, then building effective tools for yourself is key. One tool which has been very helpful for me is a PacketScanner utility which finds all the places a method and field is being used across all the classes in the client.

java

public static void findPacketCalls(Map<String, ClassNode> classes, String methodOwner, String methodName, String fieldOwner) {
        log.debug("Scanning for calls to {}.{} passing {}.*", methodOwner, methodName, fieldOwner);

        // Map to store results. Key: Packet Name (e.g., iq.ce). Value: List of locations.
        // Using TreeMap so the final output is sorted alphabetically by packet name.
        Map<String, List<String>> groupedResults = new TreeMap<>();

        for (ClassNode classNode : classes.values()) {
            for (MethodNode methodNode : classNode.methods) {

                for (AbstractInsnNode insn : methodNode.instructions) {

                    if (insn instanceof MethodInsnNode) {
                        MethodInsnNode methodInsn = (MethodInsnNode) insn;

                        if (methodInsn.owner.equals(methodOwner) && methodInsn.name.equals(methodName)) {

                            FieldInsnNode targetField = findPreviousTargetField(insn, fieldOwner);

                            if (targetField != null) {
                                // Format the packet name and the location
                                String packetName = targetField.owner + "." + targetField.name;
                                String location = classNode.name + " | Method: " + methodNode.name + methodNode.desc;

                                // Add to our grouped map
                                groupedResults.computeIfAbsent(packetName, k -> new ArrayList<>()).add(location);
                            }
                        }
                    }
                }
            }
        }

        printGroupedResults(groupedResults);
    }

Using this tooling like:

java

PacketScannerUtil.findPacketCalls(classes, "hk", "ah", "iq");

helped me find all the places the ClientPacket class was being used and gave me places to search for other key packets I needed to map. It won't map the packets for you but it does give you a good place to start following the breadcrumb trail to find the packet you need.

Stripping @Named Annotations

On the topic of building effective tooling, here is a snippet I've used to strip out all the @Named annotations from the client. It's just a simple utility method using ASM's visitor pattern:

java

   private static byte[] stripNamedAnnotation(byte[] classBytes) {
        ClassReader cr = new ClassReader(classBytes);
        ClassWriter cw = new ClassWriter(0);

        cr.accept(new ClassVisitor(Opcodes.ASM9, cw) {
            @Override
            public AnnotationVisitor visitAnnotation(String descriptor, boolean visible) {
                if (descriptor.contains("Named")) return null; // drop it
                return super.visitAnnotation(descriptor, visible);
            }

            @Override
            public MethodVisitor visitMethod(int access, String name, String descriptor,
                                             String signature, String[] exceptions) {
                return new MethodVisitor(Opcodes.ASM9,
                        super.visitMethod(access, name, descriptor, signature, exceptions)) {
                    @Override
                    public AnnotationVisitor visitAnnotation(String desc, boolean visible) {
                        if (desc.contains("Named")) return null;
                        return super.visitAnnotation(desc, visible);
                    }
                };
            }

            @Override
            public FieldVisitor visitField(int access, String name, String descriptor,
                                           String signature, Object value) {
                return new FieldVisitor(Opcodes.ASM9,
                        super.visitField(access, name, descriptor, signature, value)) {
                    @Override
                    public AnnotationVisitor visitAnnotation(String desc, boolean visible) {
                        if (desc.contains("Named")) return null;
                        return super.visitAnnotation(desc, visible);
                    }
                };
            }
        }, 0);

        return cw.toByteArray();
    }

Tips ​

Creating an Updater ​

Bytecode Instruction Shuffling ​

Write method ordering ​

Building Effective Tooling ​

Stripping @Named Annotations ​

Tips

Creating an Updater

Bytecode Instruction Shuffling

Write method ordering

Building Effective Tooling

Stripping @Named Annotations