[rnet] Simple and easy-to-use network module for raylib

raysan5 commented 5 years ago

As already mentioned several times by raylib community, a network module is a missing piece on raylib. Long ago I started drafting a possible network module API for raylib but due to lack of resources, project was stopped... I start this issue to list project specifications and references.

rnet is a simple and easy-to-use network library. Some desired features:

Available as raylib module (rnet.c) but also usable as a standalone library
No external dependencies (only the standard system libraries - winsock2.h/socket.h)
Self-contained in a single file (header-only) or two portable files: rnet.h + rnet.c
Focused on UDP protocol (probably the most useful for games)
Support WEB (WebSockets) and ANDROID platforms (if possible)

Just as a quick reference, the following APIs have been analyzed:

Unity network-transport layer - Really good API
SDL_Net - Only UDP pieces
SFML Sockets - Not bad...
Single file network libraries - Special mention for sts_net

I'm not an expert on network programming, actually last time I remember doing some sockets code was on University... but here it is basic raynet API proposal (probably very improvable):

void InitNetworkDevice(NetworkConfig config);
void CloseNetworkDevice();
int NetworkConnect(int hostId, const char *address, int port);
void NetworkDisconnect(int hostId, int connectionId);
void NetworkSend(int hostId, int connectionId, int channelId, void *data, int length);
int NetworkReceive(int hostId, int connectionId, int channelId, void *data, int length);
// Maybe hostId, connectionId and channelId can be a `struct NetworkSocket`.

Just keep in mind this API is a very rough proposal...

EDIT: Added to list desired support for WEB and ANDROID.

a3f commented 5 years ago

NetworkDisconnect and NetworkSend could both fail, they should return an error code or give the user some way to find out what went wrong (e.g. Network is down or a timeout occurred). Speaking of timeout, how will NetworkReceive work?

Blocking till a packet arrives doesn't sound too useful, so I assume it will poll for new packets and return immediately?

OvermindDL1 commented 5 years ago

Probably shouldn't use something so low level as raw tcp/udp style interfaces. Should probably build raynet around something like https://github.com/SLikeSoft/SLikeNet or so instead as they have actually useful constructs for games.

Gamerfiend commented 5 years ago

I think a NetworkSocket and NetworkData structs would be a good way to do it, with the latter having a void * ptr that the user can set as their data.

Skabunkel commented 5 years ago

One haircut later. i have some input. Aswell as questions

// This call can fail on windows, it is rare but should be accounted for.
bool InitNetworkDevice(NetworkConfig config); 

//Why is the hostid included here?
bool NetworkDisconnect(int hostId, int connectionId); 

//This should return something, maybe not int but atleast a bool. 
//But giving the user an option to track the sent bytes is useful
//Why is hostid included here?
//What is channelId supposed to be used for?
int NetworkSend(int hostId, int connectionId, int channelId, void *data, int length); 

//Why is hostid included here?
//What is channelId supposed to be used for?
int NetworkReceive(int hostId, int connectionId, int channelId, void *data, int length);

Other than that i can only suggest expanding it with some encoding functions etc.

Skabunkel commented 5 years ago

I'll make this a separate thing, focusing on UDP and implementing WebSockets seems sort of weird? I might have a different notion of what a WebSocket is in that case please explain.

Provided that the idea is to be able to make a GET and POST request this should not be impossible, however then we should sit down and focus on what part of the HTTP protocol, we would want since posts and gets can be very funcy if you let them.

Skabunkel commented 5 years ago

Probably shouldn't use something so low level as raw tcp/udp style interfaces. Should probably build raynet around something like https://github.com/SLikeSoft/SLikeNet or so instead as they have actually useful constructs for games.

Raylib is written in c99 so this library could not be used if we want to keep it that way.
See bullet point 2 in rays post.

OvermindDL1 commented 5 years ago

Raylib is written in c99 so this library could not be used if we want to keep it that way.

It itself is yes, hence why it should be entirely wrapped, it should not be exposed to the user of the library at all.

No external dependencies (only the standard system libraries - winsock2.h/socket.h)

It's license allows baking it into a project so it's no longer an external dependency.

raysan5 commented 5 years ago

Thanks to all of you for your answers! :)

@a3f @Skabunkel yes, functions should return some state info. @a3f don't know yet how those functions should properly work. @Gamerfiend agree. @Skabunkel hostId is probably not required, I took it from Unity network API. Some basic encoding function could also be useful to encode/decode packages. About WebSockets, I understand is the equivalent of sockets to run on web, maybe I'm wrong with my assumption. @OvermindDL1 Thanks for the sugestion but that's a big library, I prefer to keep it simple and self-contained.

Skabunkel commented 5 years ago

Ah, that makes sense. unsure how that would work in webasm but oh well, the joy of exploration. Edit 1: https://github.com/emscripten-core/emscripten/tree/incoming/tests/sockets Edit 2: Probably more this https://github.com/emscripten-core/emscripten/blob/incoming/tests/websocket/websocket.c

Skabunkel commented 5 years ago

Android NDK sockets https://github.com/wzbos/Android-NDK-Socket/tree/master/sample/app/src/main/jni/tcomm.c This is just reference material, does not seem to scary based on it. But i have to admit i do not read chinese so i might be wrong. 😄

zakrent commented 5 years ago

Why so low level? Some things like packet sequencing and relability would be nice (something like eg. ENet, look at packet flags), those things are hard to make for new programmers and would help them with networking.

Syphonx commented 5 years ago

@zaknet I've been working on small network module for raylib - hence why this issue was opened, I plan to have a similar api to sfml/sdl_net for those who want to use platform agnostic socket calls.

I've also got plans to include a, as you put it 'higher level' game api on top, the goal is to support basic replication, serialisation, callbacks, sequencing etc for those less familiar with networking

Syphonx commented 5 years ago

With raylib having quite a minimalist api however, my roadmap for the module is very much subject to change. Within the week I'll likely post a spec, outline what features I intend to add and we can have a discussion about what belongs in an api call, and what we can provide as examples :)

raysan5 commented 5 years ago

@Skabunkel Thank you very much for the links! I looks pretty good! Supporting WEB and ANDROID has been added to the list! @zakrent I agree, intention is to offer a simple and easy-to-use API but I'd start with a bare-bones working solid version first. @Syphonx I'm following your rnet implementation, you're doing a great work! I think you're using TCP at this moment, are you planning UDP support also? We can discuss it later on once specs published... Keep up the good work! :)

raysan5 commented 5 years ago

By the way, @Syphonx, do you think we could have a functional version to be included on raylib 2.5 in the following two months or so?

Syphonx commented 5 years ago

I plan to have barebones support for TCP/UDP, a low(er)-level socket api, serialisation helpers and some examples done in the next week or so @raysan5

Syphonx commented 5 years ago

Features like replication, data packet tags and more advanced features will likely be added on an 'as we go' basis

nxrighthere commented 5 years ago

An attempt to use any networking library that includes winsock2.h on Windows, will result in name colliding and conflicts with Raylib functions and structures at the compilation, specifically: Rectangle, UnhideWindow, CloseWindow, and ShowCursor.

Windows headers already have these declarations. I'm surprised that you don't follow an appropriate naming conventions with a vendor-specific prefix, this is quite trivial thing in C.

nxrighthere commented 5 years ago

I used Raylib in NetDynamics if you're interested. I've added Ray prefix to all functions and renamed Rectangle to Rectangle2D. Without these modifications, it's impossible to build a project on Windows with Raylib and included winsock2.h.

raysan5 commented 5 years ago

@nxrighthere Hey! Excuse me for this super-late response, I completely forget about this issue...

About windows.h inclusion (or related headers), symbol collisions could be mostly avoided like explained on #857, also embedding the inclusion into its own module.

Personally I don't like prefixes (when I started raylib I came from C# world) but I agree it's the best option in C to avoid that kind of collisions... maybe in a future update of raylib...

Congrats for the NetDynamics, very nice project! Great code organization and github project presentation. I like it!

nxrighthere commented 5 years ago

@raysan5 Thanks, I glad that you like it.

Personally I don't like prefixes (when I started raylib I came from C# world) but I agree it's the best option in C to avoid that kind of collisions... maybe in a future update of raylib...

Well, it's not a problem for me I've just made a small script that renames everything. But, for example, people to whom I suggested Raylib in my community encountered the same problem, and they were really unhappy with that, so I think it would be better to solve this out of the box with the traditional approach.

Thank you for your work anyway, it's a very helpful thing for fast prototyping.

raysan5 commented 5 years ago

Just for reference, I found swrap library, looks very interesting!

nxrighthere commented 5 years ago

Such stuff is just a very basic UDP API, for multiplayer games, you will need a networking library with a mature protocol that carries at least these features:

Secure connection association and management (through cryptography preferable: https://github.com/jedisct1/libsodium or https://github.com/jedisct1/libhydrogen for example)
Efficient reliability to guarantee delivery of data (see https://github.com/skywind3000/kcp for example)
Flow and Congestion Control (should be pluggable and optional, not every project will benefit from it)
Packet fragmentation and reassembly, IP fragmentation should be avoided in any case
Packet multiplexing to efficiently aggregate data when possible

This is only the tip of the iceberg, consider something that provides such feature-set out of the box. Godot is using ENet which is used in many modern games today such as League of Legends for example (they are using cryptography for payload on top).

lmnt-dev commented 4 years ago

For inspiration: https://github.com/networkprotocol/yojimbo

paezao commented 4 years ago

Hey guys! Is someone working on this?

raysan5 commented 4 years ago

Hi @paezao! Not really, I did some review a while ago but it's not functional at this moment...

My plan was simplifying the API and use UDP-only.

masterneme commented 4 years ago

Hey @raysan5, I asked you recently about enet. You told me that you're looking for single files with single headers.

But do you actually like enet? I mean, if someone took the source code and put it all together in a single organized file+header, would you use it or does enet lack features that you want?

nxrighthere commented 4 years ago

A single-header fork is available here. It's used in several games in production.

raysan5 commented 4 years ago

@nxrighthere Oh! Thank you very much for sharing!

masterneme commented 4 years ago

What a coincidence @nxrighthere , I recently starred your fork before posting here, I hope @raysan5 likes it. I want to use it for some Unity projects.

raysan5 commented 3 years ago

Another great alternative: enet single-header fork: https://github.com/zpl-c/enet

bloodhaze13 commented 3 years ago

zpl-c/enet is the particular repo that I used in my multiplayer pong clone I wrote a couple of days ago. I have little to no background in C and managed to get it up and going fairly quickly which suggests it must be fairly easy to use.

Client snippets: https://pastebin.com/6Y84AzRd
Server snippet: https://pastebin.com/XBwXGeZX

Rough mapping of code snippets to proposed API below:

- InitNetworkDevice()/CloseNetworkDevice() -> Handled by ENet in GameServerConnect()
- NetworkConnect() -> GameServerConnect()
- NetworkDisconnect() -> GameServerDisconnect()
- NetworkSend() -> GameServerPublish()
- NetworkReceive() -> GameServerPoll()

So a thin wrapper over ENet within the same single file seems viable, so long as your happy with the ENet license and whatever maintenance goes along with it. The only other convenience function that would be nice is some sort of transport-layer compression and encryption which should be possible before crafting the ENet packet.

I started having a look at the existing rnet code and for a novice like me it looks like it would be a little easier to maintain/debug than ENet since it is much simpler. That said there would probably be more work in completing it than an ENet wrapper. Effort/maintenance/functionality tradeoff I guess.

raysan5 commented 3 years ago

@bloodhaze13 thanks for the info! Current rnet implementation was contributed by an user following a proposed API but development was stopped, I tried to pick it up but it required more time than expected, I'm not an expert on networking...

Lately I also keep an eye to another very promising library: https://github.com/nathhB/nbnet

vb-mich commented 3 years ago

Hello, for an old project of mine I used this "Tiny Websockets" library https://github.com/Theldus/wsServer It's works pretty well and quite RayLib-style (just a couple of .c file, written in C99), the only thing is that it doesn't support encryption yet, even tho there's an easy workaround for that. [edit] Oh, I just noticed that @Theldus actually forked RayLib as well XD

raysan5 commented 3 years ago

This issue has been open for too long, many alternatives have been proposed here and actually rnet.h implementation is still there for anyone willing to review/improve it. I'm closing the issue.

vb-mich commented 3 years ago

Mongoose Embedded web server seems to be a complete embedded and self contained web server written in C and shipped in a sigle C file (+header) https://cesanta.com/docs/#build-options

I drop here this reference to keep track of it.

nathhB commented 3 years ago

Hello.

I'm the author of nbnet. I discussed with Ray and started working on a potential rnet implementation (as a nbnet wrapper). You can follow the progress here for now. Take a look at rnet.h for a good overview of the API.

raysan5 commented 3 years ago

@nathhB I'm reopening the issue for further discussion!

nathhB commented 3 years ago

@raysan5 I forked raylib: https://github.com/nathhB/raylib.

This is where I will update rnet so you can look there when you want to review it. I'm gonna leave it as-is until it has been reviewed as I think it lays a good fundation to build upon. There are two files: rnet.h and rnet.c.

raysan5 commented 3 years ago

After reviewing rnet implementation using nbnet I got to the conclusion that nbnet is so good that a rnet wrapper does not make much sense. I checked the code and almost all rnet functions just implement a direct call to some nbnet function, maybe adding some tracelog or small check.

Other libraries used on raylib like miniaudio are more complex and having a top layer simplifying usage makes sense but nbnet already seems very easy to use, even for beginners.

Still, some nbnet usage examples will be added to the official raylib/examples/network and nbnet will be the go-to option recommended for raylib users wanting to add network features to their games. :D

raysan5 / raylib

[rnet] Simple and easy-to-use network module for raylib #753