Shared Networking Library

[!NOTE] This is a concept of a future library with the intent of sharing packet encoders and decoders for all private servers.

Prerequisites

This library would depend on Netty 4.x and utilize Netty's buffers.
This library would be written in a recent version of Kotlin that is to be determined in the future.
This library would depend on experimental features such as value classes.

Goals

Utilize Jagex's naming for packets, buffer functions, properties and such wherever possible. Educated guesses will be made for any missing names.
In a given revision, every single packet on a given platform should be implemented, even if servers may not end up using them.
Provide a memory-efficient way of handling most of the smaller packets, such as IF_OPENSUB. Primitive value classes can be utilized if all the values fit into 8 bytes or less. The lowest primitive type that can carry the payload would be utilized. The gains received from bitpacking the properties is still received, though.
Provide an efficient and correct way of implementing PLAYER_INFO and NPC_INFO packets. Most servers do not utilize these packets correctly, leaving a lot of performance on the table. These packets are very complicated and will need lots of discussion and documentation on their own.
This library will prioritize performance over "clean code". Low-level optimizations such as loop unrolling may be used if it provides noticeable performance improvements. Bitpacking will be performed where it has benefits.
Provide a way to opt-out of decoding specific packets that a server does not wish to utilize. An example of this could be EVENT_MOUSEMOVE. It is a fairly active packet and will consume unnecessary CPU power and memory to decode it, which would all go to waste if the server discards the values.
Provide a way for the end user to define their preferred method of providing buffers. Additionally provide an efficient way of handling buffers with the library itself, utilizing pooled buffers for var-size packets, and unpooled buffers for fixed size packets.

Non-Goals

This library would NOT contain all revisions. Versions would be made on-demand by contributors, it's very possible there will still be large gaps.
This library would NOT provide networking configuration, JS5 implementations or any other steps related to networking, it will only deal with game packet decoding and encoding.

Motivation

Private server developers targeting the OldSchool builds are often spending days re-implementing the same packets that several others have done before them. We are often reinventing the wheel in the process too, coming up with various, often sub-par implementations of the protocol. While this won't be the easiest project to pull off, it would go a long way to simplifying the process for everyone involved, while also catapulting the OSRS protocol much further by lowering the skill requirement needed to enter the OSRS scene. Overall, this should significantly reduce the amount of time developers are spending on figuring out the packets each revision, as instead of everyone having to figure it out, it's only a single user that has to figure it out.

Protocol Constants

Protocol constants, such as the opcode and size of the packet, will be represented through an enum. The opcodes will be separated out into an internal object - this is to avoid having to scramble the enum every revision. Separating them allows us to have clean version history to see how packet sizes change over time, as well as any new additions or removals. It will also allow us to nicely group the constants according to their category. The enum containing opcodes will effectively be scrambled every revision, as it will be ordered according to the opcode, as it is found in the client.

Example of ServerProt:

[!NOTE] ClientProt will be identical to ServerProt, only with different constants.

internal object ServerProtId {
    const val IF_OPENSUB = 0
    const val MESSAGE_GAME = 1
}

enum class ServerProt(
    val id: Int,
    val size: Int,
) {
    IF_OPENSUB(ServerProtId.IF_OPENSUB, 7),
    MESSAGE_GAME(ServerProtId.MESSAGE_GAME, VAR_BYTE),
}

Messages

Each incoming and outgoing packet would be represented as a POJO-like message. These messages will be exposed to the server. Bitpacking will be utilized where possible to reduce the footprint of these classes.

All messages will implement a simple binding interface:

interface Message
interface IncomingMessage : Message
interface OutgoingMessage : Message

Example message:

data class MessageGame(
    val message: String,
    val targetUsername: String?,
    val type: Int,
) : OutgoingMessage

[!CAUTION] The developer is responsible for ensuring validity of the data passed into the constructors. In the above case, the type property is actually encoded as a single byte, if the developer provides a value outside of its boundaries, it will simply be subject to sanitization through bitwise-and (in this case, & 0xFF). The only exception to this is smarts, where the data must be within valid boundaries or it will throw an exception.

Message Encoding

Messages will be encoded through a MessageEncoder interface. This interface will be public and developers can create custom implementations of packets that they may have modified, rather than needing to fork the entire repository. Implementations of the packet encoders supplied by the library will be internal though.

interface MessageEncoder<in T : OutgoingMessage> {
    fun encode(
        buffer: ByteBuf,
        message: T,
    )
}

Examples of message encoders:

internal class IfOpenSubEncoder : MessageEncoder<IfOpenSub> {
    override fun encode(
        buffer: ByteBuf,
        message: IfOpenSub,
    ) {
        buffer.p2(message.interfaceId)
        buffer.p4(message.bitpackedTargetComponent)
        buffer.p1(message.type)
    }
}

internal class MessageGameEncoder : MessageEncoder<MessageGame> {
    override fun encode(
        buffer: ByteBuf,
        message: MessageGame,
    ) {
        buffer.pSmart1or2(message.type)
        val name = message.targetUsername
        if (name == null) {
            buffer.p1(0)
        } else {
            buffer.p1(1)
            buffer.pjstr(name)
        }
        buffer.pjstr(message.message)
    }
}

Message Decoding

Messages will be decoded through a MessageDecoder interface. This interface will be public and developers can create custom implementations of packets that they may have modified, rather than needing to fork the entire repository. Implementations of the packet decoders supplied by the library will be internal though, much like with encoders.

interface MessageDecoder<out T : IncomingMessage> {
    fun decode(buffer: ByteBuf): T
}

Examples of message decoders:

data class IfButton3 internal constructor(
    private val _interfaceId: UShort,
    private val _componentId: UShort,
    private val _slotId: UShort,
    private val _objId: UShort,
) : IncomingMessage {
    val interfaceId: Int
        get() = _interfaceId.toInt()
    val componentId: Int
        get() = _componentId.toInt()
    val slotId: Int
        get() = _slotId.toIntOrMinusOne()
    val objId: Int
        get() = _itemId.toIntOrMinusOne()
}

[!IMPORTANT] Packet properties will utilize the lowest possible data type due to 64-bit JVM memory alignment. This effectively means all classes will be aligned to a multiple-of-8-bytes, therefore it is still beneficial to use lower data types, such as in the above case, as the memory footprint will be lower. Packets which are naturally smaller, e.g. one that might only write a single byte, can still be represented in its integer form, as it will be aligned to 8 bytes anyhow. There are no benefits to be had from using value classes due to the guaranteed autoboxing occurring at multiple turns (generics, interfaces). If the class gets autoboxed, it will also be subject to the memory alignment, thus eliminating any benefits.

blurite / rsprot