Rust implementation

[vvanders]

I've started on a wrapper of netcode for Rust located at https://github.com/vvanders/netcode-rust (until we can figure out best way to merge).

Right now only client functions are hooked up and nothing is tested but it's a bit of a start. Currently it bootstraps netcode via "gcc" crate which shells out to msvc pretty cleanly on windows. Still need to sort out linux.

I'll update this issue once I've got something more stable, this is mostly a placeholder to discuss ongoing integration and related items.

[gafferongames]

I tested this, and found that with this code:

    // read variable length sequence number [1,8]

    int i;
    for ( i = 0; i < sequence_bytes; ++i )
    {
        uint8_t value = netcode_read_uint8( &buffer );
        (*sequence) |= ( uint64_t) ( value ) << ( 8 * i );
    }

It does what I expect, which is to have in memory (in register), an 64 bit sequence number in the correct order.

You should be able to use code that handles this, by running this same logic, and then converting from big endian.

Alternatively, you can process it another way, but the reversed bytes written shouldn't stop you from handling this any way you wish.

cheers

• Glenn

[vvanders]

Sorry it took me a while to reply back, I definitely can adapt the code to what C does.

I guess what my initial issue was if libsodium treated the nonce as a native integer or as a series of bytes so that we'd have to take endian ordering into account when serializing. I did some more digging and from what I can tell it's endian agnostic so I'll just go ahead and match the C API.

[gafferongames]

I think you're on to something with the way sequence numbers are fed into libsodium.

Yes, libsodium actually works on byte arrays, but in the C implementation I was aliasing the byte* to the uint64_t sequence # pointer. This meant that on little endian platforms, the byte arrays were in little endian order in memory, and in big endian platforms, byte arrays in big endian order.

So we need to fix this. I recommend standardizing on feeding libsodium little endian order byte arrays, because that's what we do everywhere else. So in short, the only change that needs to be made is to check if big endian and byteswap the sequence number before calling libsodium functions.

I'll make this change in the C reference implementation tonight.

[gafferongames]

Ah my memory is so rusty, this is already taken care of:

uint8_t nonce[8]; { uint8_t * p = nonce; netcode_write_uint64( &p, sequence ); }

[gafferongames]

So basically, make sure the sequence number passed in as nonce is little endian order.

I'll update the standard to make this clear.

[vvanders]

Hmm, that's weird because I tried both little endian and big endian serializations, neither worked(I'm sure the serialization is good because I can read/write other fields between C/Rust).

Anyway, let me sort out exactly what works with the latest HEAD(I just synced a bit and have most things working) and I'll report back.

[vvanders]

Also, do we really need to be doing any of these transforms or can we just say that the top bytes of the sequence are padded zero and use what's on the wire as-is? Seems like it would be even simpler since we don't care about the actual numeric value, just that it doesn't repeat.

[vvanders]

Actually, nevermind. Just updated to latest and have it working now.

That's what I get for trying to get things done piecewise. Sorry for the churn, should hopefully have the last of the serialization done soon and then might have clients talking to the server.

[gafferongames]

No worries. Sounds great! Keep up the good work

[gafferongames]

Heads up, I'm almost finished with the standard here: https://github.com/networkprotocol/netcode.io/blob/master/STANDARD.md

I've finished everything except the server-side processing packet section, and as I think through the handling of the connection response packet I think I've identified a possible attack that can be made, so I'm going to think this over today and probably add a few additional checks.

I'll keep you up to date on this.

[gafferongames]

Oh wait, actually the additional checks aren't required because the encryption mapping times out automatically after 5 seconds. Nevermind. This means I can remove the HMAC from the challenge token data... I'll make this change now.

[gafferongames]

OK the change has been made in the standard and the reference C code. Please update.

[gafferongames]

I've made a change another change to address a potential vulnerability. When the server adds an encryption mapping for a potential client, that encryption mapping not only times out if no packets were sent or received from that address for timeout seconds, but also if that address fails to establish connection with the server within timeout seconds.

This fixes a vulnerability where an attacker could get a connect token, establish an encryption mapping, and then spam the server forever with large packets without establishing connection. This would allow a "piling on" of multiple spamming fake clients over time, which could not be detected or kicked by any backend process, because they don't progress to connected. Now any client that tries this is kicked in timeout seconds, which should eliminate the possibility of piling on.

This changes is reflected in the standard and the C reference implementation.

See 6170e478c1020e1776c2c471400b9e1008f4d50b and dda5563ad97ff53fd007f458d6d105632f8ee3b9

[gafferongames]

I also noticed that the user data passed in from the connect token -> challenge token was not being stored on the server, so I added code to the C reference implementation to cache the user data per-client, and an accessor function to query it: netcode_server_client_user_data

See 87e5c0fe203bc49231b8a85bfcc61a6f500b5002

This should be it for changes. I'm going to see if I can finish the last part of the standard tonight.

cheers

[vvanders]

Sounds good, I'll try and take a look at it in a little bit. I'm still working on getting the server doing the basic auth route but I'll make sure to include that when I get to the security checks.

[gafferongames]

Standard is now complete, I think.

Please let me know if you think anything is missing or if anything is not clear:

https://github.com/networkprotocol/netcode.io/blob/master/STANDARD.md

[vvanders]

Still plugging away at establishing a connection(I feel like I've been saying I'm getting close for a week now).

One quick, probably dumb question. Is there any reason you're using a different crypto primitive for encoding the challenge packet from normal packets? It's not too hard to add but seems a bit inconsistent.

[gafferongames]

It's historic.

Previously the packets were encrypted with this simplebox primitive too, but then I realized they needed the additional data for versioning and safety (eg. to make it impossible to change a packet type in the header and so on).

The connect token needs additional data because it has the private and public portions.

But a challenge token is only private data, so it has no additional data. Hence it stayed with the simplebox primitive.

cheers

[gafferongames]

Would you like to create a pull request for your latest work into this repo? Having it in here would get it a lot more visibility. I'd be happy to accept pull requests as you reach milestones or when you have made progress you want to share.

[gafferongames]

ps. There seems to be somebody working on a C# implementation as well in parallel with you, so you should have company shortly.

[vvanders]

Yeah, that'd be great. Let me mark all the in-progress stuff as private and I'll get something put together tomorrow.

Any chance to unifying the crytpo? crypto_aead_chacha20poly1305_encrypt() lets you have the additional data as optional(I have it mapped to Option<&[u8]> on the Rust side) so we could use a single function for both paths.

Glad to hear there's a C# implementation starting up, who knows they might even beat me to completion :).

[gafferongames]

I think we can unify the crypto. I'll work on that today.

[gafferongames]

Closing this issue out. Creating a project for discussion on the Rust implementation

[gafferongames]

Encryption is unified in reference implementation, standard updated.

[vvanders]

Huh, no idea why I didn't see the project stuff before, neat!

Thanks for unifying the crypto. I'll update to latest and continue onwards towards getting the server running.

[vvanders]

Also looks like I don't have permissions to access the project board, if you want to give me access I'll throw up a few tasks for the stuff I've been looking at.

[gafferongames]

I'll see if I can find out how to give you access to that

[gafferongames]

Until, then re-opening!

[gafferongames]

I've tried adding TODO and In Progress columns, but I'm not sure if you can edit them or not.

Not sure if I like the projects feature or not yet...

[gafferongames]

Very small change made as I did another pass over the standard today:

8920fde7240b462a39895374edb512c237af7b7a

If the connect token create timestamp is greater than the expire timestamp, it should fail to read.

cheers

[vvanders]

Yeah, I can't edit or anything yet. I saw the collab invite but didn't seem to work.

Not an issue, I'm happy to keep using this and submitting PRs.

One quick Q, I've got cargo + doc links in the from the old rust readme, mind if I merge that over into the main readme here?

[gafferongames]

Hello, I have recently made some small changes to the C implementation to support custom allocators, but it has broken the Rust impl. Also, in my own testing with another project, I found that if UDP sockets are created with IPv4 rather than IPv6, then they will work when run under Travis.

[gafferongames]

Sorry, I added const support to the C API (see NETCODE_CONST) but it seems to have dorked up Rust again. On the positive side, you now have a way to mark bits you want const as const, I remember these were causing issues for Rust type safety...

[gafferongames]

I've had a crack and fixed most issues, but one test is still disabled (needs to be updated to pass in a null allocator context and function...), plus there is some weird stuff that looks like the private_key passed in is never used. Also, bunch of complaints about variables, functions and structs that aren't used. Would be nice to clean those up, but I don't want to be a bull in a china shop here ;)

[vvanders]

By all means feel free to edit/modify/improve, I don't mind at all. In fact I'm going to be pretty slammed at work for the next few months so I don't think I'll be able to get it anytime soon.

[gafferongames]

BRB learning Rust...

[gafferongames]

Heads up, I just don't have the headspace to maintain Rust, Golang and C versions when I make small changes. Because of this think it's best that Rust lives as it's own repo from now on. I will continue to keep golang and C impl in the main repo because I can actually maintain these myself.

cheers

• Glenn

[rohel01]

Hello, the rust implementation repository seems gone :(

[gafferongames]

Yes, we have decided to keep this repository for the C reference implementation only.

You can find the Rust implementation in its own repository now:

https://github.com/vvanders/netcode.io

If somebody would like to fork this and continue development, that would be great. It doesn't seem to be maintained anymore.