reduz commented 3 years ago

Describe the project you are working on

Godot

Describe the problem or limitation you are having in your project

With the rise of popularity of monetizable multiplayer games, more users want to work on this. These types of games, however, require dedicated authoritative servers (hosted on the cloud via Kubernets, Agones, etc), to avoid users from cheating (And hence ruining the monetization).

Godot seems to work well for this due to its Linux friendly and Open Source nature, and the fact it can run in headless mode. There are some hurdles to overcome, however to make this process smoother for users:

Exports still include all graphics and audio assets, which get included even if exported to server.
Exports still include animation (and skeleton) data, which can be expensive to process even if there are no graphics.
Export still includes all scripts, and its not possible to discern what is not needed on the server (or in exchange, what is, since you may want to include information to connect to a back-end that is not present in the player side).
Several nodes may simply be undesired in the server (as an example, Sprite, MeshInstance), so at least it should be possible to run logic to strip nodes on server export by marking them for this end.
Running a game at lower FPS to maximize CPU usage in a container (this is handled by #2753 though).

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Server export should take care automatically (for the most part) of all the points above.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

The server export should be modified to include the following:

Some nodes in the scene should be possible to mark as client-only (will be erased on server export, and subsequently also their animation data or references to them).
Animations should be able to mark tracks as server-only, so the rest are stripped.
Skeletons should be able to mark bones as server only, so the rest are stripped (remember, these can still be useful if root motion is used, or bones control physics objects via BoneAttachment).
Graphics and Audio asset stripping, replaced by dummy assets:
- Textures will be replaced by a DummyTexture that does nothing except report their correct size (if still used, remember some nodes may require this size for something).
- Same for meshes, a DummyMesh will report the right AABB but will contain no data.
- Streams will be replaced by DummyStream, which will do nothing, only report the correct length.
An Engine.is_dedicated_server() method should be added for scripts to check.
It should be possible to mark server assets before exporting as:
- Strip: Graphics and Audio assets are stripped on server. (default)
- Keep: Graphics and Audio assets are kept on server.
- Remove: Asset will simply be removed on server.
- RemoveOnClient: Asset will be removed on non-server exports.

If this enhancement will not be used often, can it be worked around with a few lines of script?

Is there a reason why this should be core and not an add-on in the asset library?

Users are already using hacks to do this, but its far from easy to use and unoptimal, or require source code tweaking at engine level. This feature needs to be core.

Short FAQ:

Q: I still need access to the pixels of an image on the server, what do I do here? A: Remember that in Godot images can be imported as Image rather than Texture, so by default those will not be stripped, as there is not much reason to. In the end you will still be able to mark the asset as "Keep" so its not stripped.

WolfgangSenff commented 3 years ago

This would be really, really useful for @beardooks and us GodotNuts generally, because he maintains a Docker script that can point to a git repo and automatically exports the headless option and sets up multiplayer for it on a server, opening up ports and such. As such, having the extra option would make that script create far better server-side exports.

Whimfoome commented 3 years ago

So are we moving to one game project for both server and client, similarly to UE4? Because before we had to make 2 separate projects (at least in Game Development Center's tutorial series approach).

reduz commented 3 years ago

@Whimfoome Officially dedicated server exports were never really supported. The Godot server build was mostly for doing CI exports and tools, not to run as dedicated multiplayer game server. This is something Godot users discovered they could do pretty well so from our side the idea is to support it as best as possible.

aRestless commented 3 years ago

This is great news!

I have a background in game server hosting and would like to add a few topics to look out for. Some of them might already be on your radar or already solved by Godot. Some might also not fit the scope of this issue and could warrant separate ones:

Allow IP binding and port binding for all allocated network sockets, meaning exposing these values as configuration parameters; Many game server applications do not support this, which can force game server providers to build custom hooks for the binaries to overwrite socket allocation
For any functionality related to game server lists, e.g. the dedicated server registering itself towards some backend, do not rely on the server listening on the same IP that the request comes from (the default/gateway IP might be different to the one the server is bound to)
Add an option to limit server FPS while no player is connected; this allows overcommitting CPU resources
Make sure the engine properly handles environments with typical server specs, such as 48 physical CPU cores across two CPUs; e.g. Unity has some standard functionality that spawns one thread per core, which is not desirable in these environments.
Support a standardized "server query" method for information like connected player numbers, current map, version, and so forth; Steam's A2S Query is widely used even outside Steam's ecosystem. A very bare-bones reference implementation can be found here: https://github.com/nitrado/libSteamProt

I hope this is helpful.

YuriSizov commented 3 years ago

reduz commented 3 years ago

@pycbouh Not technically related I guess since one thing does not depend on the other, but definitely it needs to work for everything to make sense.

reduz commented 3 years ago

@aRestless I have the feeling that most of these can be done with add-ons (either official or not) since they are mostly game specific. Regarding hosting, Godot seems to have some things that are different than other engines, such as being very efficient resource wise, very quick to spawn and load, and you have very precise control on how threads are used. I guess as more users use it for this, we will understand better what can be optimized.

Calinou commented 3 years ago

Allow IP binding and port binding for all allocated network sockets, meaning exposing these values as configuration parameters; Many game server applications do not support this, which can force game server providers to build custom hooks for the binaries to overwrite socket allocation

Setting the server port can already be done, but it's up to you to expose it with a CLI argument (OS.get_cmdline_args()) or configuration file (ConfigFile class).

I think the listening IP can be set too, but I'm not sure (cc @Faless).

Add an option to limit server FPS while no player is connected; this allows overcommitting CPU resources

This can already be done by setting Engine.target_fps and Engine.iterations_per_second to a low value (try 1). Once a player connects, set those to 0 and 60 respectively (or whichever values you were previously using).

aRestless commented 3 years ago

@reduz I absolutely agree that some points in the list are more additional features and could be suited for add-ons.

@Calinou Thank you for those insights. IP/Port binding is probably more a documentation topic then. I have supported a number of game launches where, quite frankly, game developers were not even aware of the fact that a system can have more than one IP address and the missing support for IP/Port binding resulted in significant road blocks for going into production.

Speaking from a provider perspective, I'd love for IP/Port binding always being something explicit and configurable from the outside (instead of "0.0.0.0" from a code sample being hardcoded and no one ever going back to it). But such an approach might not fit into Godot's design.

BearDooks commented 3 years ago

So are we moving to one game project for both server and client, similarly to UE4? Because before we had to make 2 separate projects (at least in Game Development Center's tutorial series approach).

@Whimfoome I was able to use one project for this and then used the 'features tags' so I could make functions on work if it was running as the 'client' or the 'server'. You still need to export the project twice, once with each tag.

@reduz I think this would be a wonderful addition to the engine. As you have stated before it is not always easy to run a multiplayer headless server without some hacks to it. In my personal opinion I think it would allow more people to create multiplayer experiences in a less frustrating manner.

nonunknown commented 3 years ago

@reduz I have a question also, My job requires me to do a multiplayer game using godot, Currently I'm not using the godot itself to do server stuff, but instead only using WebSocket do pass data, and protobuffers to reduce its size (which a guy made a plugin to support protobuffers in godot) https://github.com/oniksan/godobuf . Can this be official?

The second question is, what about having access to godot's mainloop via API, I mean My server uses GoLang (this can be true for every language) So I could have godot's mainloop running in the server and call stuff from there, and replicate to clients, its possible?

PS: Currently I'm using raylib on the server to simulate the game there, and the client (godot instance) will only replicate whats happening on the server.

LillyByte commented 3 years ago

I would say, make sure this also supports Windows, as I only see Linux mentioned.

I know when people think about "dedicated hosting" they often think of that little cloud server out there that's running Linux-- but that's not how it works most of the time.

The bulk of gamers are Windows users; and a lot of users will launch dedicated servers on their home machines so they can play with family/friends without having to rent/manage servers.

Calinou commented 3 years ago

I would say, make sure this also supports Windows, as I only see Linux mentioned.

This will be supported once https://github.com/godotengine/godot-proposals/issues/991 is implemented.

Note that you can already run dedicated servers on Windows by passing the --no-window command line argument, even on a Windows Server machine (since it always has a GUI subsystem available, unlike Linux).

aaronfranke commented 3 years ago

@reduz I know you already said that it's not related, but really this proposal does not make sense to do until after #991. The ability to run a server headless seems more important than the ability to optimize the build by removing textures etc.

EDIT: #991 is now implemented.

Calinou commented 3 years ago

@reduz I know you already said that it's not related, but really this proposal does not make sense to do until after #991. The ability to run a server headless seems more important than the ability to optimize the build by removing textures etc.

While this proposal is not very useful without #991, these proposals can be implemented independently from each other (see above). Rest assured that #991 will be implemented :slightly_smiling_face:

See discussion starting from here: https://chat.godotengine.org/channel/networking?msg=4SuqcdnjRfY83NhSk

reduz commented 3 years ago

@aaronfranke It's unrelated, because you will be able to run a server export on any target. It's also easier to do this proposal first, as the second requires APIs to be closer to final to do best.

silverkorn commented 3 years ago

This feature will be great but I personally value more the https://github.com/godotengine/godot-proposals/issues/991 to be honest. Workarounds are possible here but it's not really possible with #991.

I will sound conspiratorialist but this proposal's score feels biased/unfair against the other one since it was created and published all around social medias by the co-creator, where unfortunately #991 was kept dormant on a quite high rank (~92 / 1826 proposal) that would be even more if it was not superseded (Lost a part of the previous scores)...

991 (June 2020)

I know it's somehow unrelated to #991 but the wording of this issue's title made me thought there would finally be some work on the cross-platform headless/server mode...

Anyway, sorry for my overwhelm about this 😅

TL;DR: Please don't let #991 die and I really hope it will be done along this one!

AlexDarigan commented 3 years ago

So are we moving to one game project for both server and client, similarly to UE4? Because before we had to make 2 separate projects (at least in Game Development Center's tutorial series approach).

@Whimfoome Are you talking explicitly about creating two seperate projects for the purpose of exporting or more generally about desiging the server and client at the same time? Because the latter became possible in one of the 3.2 updates. You can find an example here https://github.com/AlexDarigan/CustomMultiplayerGodot. (Though having a node that defaults this behaviour including to its children might be a good seperate proposal).

* Allow IP binding and port binding for all allocated network sockets, meaning exposing these values as configuration parameters; Many game server applications do not support this, which can force game server providers to build custom hooks for the binaries to overwrite socket allocation

* For any functionality related to game server lists, e.g. the dedicated server registering itself towards some backend, do not rely on the server listening on the same IP that the request comes from (the default/gateway IP might be different to the one the server is bound to)

* Support a standardized "server query" method for information like connected player numbers, current map, version, and so forth; Steam's A2S Query is widely used even outside Steam's ecosystem. A very bare-bones reference implementation can be found here: https://github.com/nitrado/libSteamProt

@aRestless I'm not an Industry Professional but I've worked with Godot Multiplayer a bit.

On your first point, I don't think this would be hugely difficult to implement as a user. If we're talking about just taking an IP or Port we know and having godot server point to it. Godot also has the IP Class that has the useful get_local_addresses function.

On your second point, for high level multiplayer, you can change the listening port using the set_bind_ip function.

And I do support the idea of a standarized servery query. Connected Players is technially achievable using get_peers().size()

vybr commented 3 years ago

Would there also be the possibility to give the headless console a command line? This is so users can input server commands to affect the game (e.g. starting/stopping the server, spawning items, banning users). Not sure how useful that is for dedicated servers, but for player-hosted ones (e.g. Minecraft, Terraria) where servers are commonly hosted on the player's own machines, it's nice to have.

Calinou commented 3 years ago

Would there also be the possibility to give the headless console a command line? This is so users can input server commands to affect the game (e.g. starting/stopping the server, spawning items, banning users). Not sure how useful that is for dedicated servers, but for player-hosted ones (e.g. Minecraft, Terraria) where servers are commonly hosted on the player's own machines, it's nice to have.

This needs https://github.com/godotengine/godot-proposals/issues/2322 to be implemented. Once that proposal is implemented, this command input feature can be implemented by an add-on :slightly_smiling_face:

In the meantime, you could make the server read from a text file every so often (every 10-30 seconds, possibly more often if a command was recently input). If the file isn't empty, execute its contents as a command and remove the file's contents (but do not remove the file entirely). With modern text editors reloading files automatically when they change on the filesystem, the user experience won't be so bad (while still not being great).

Another alternative is to use the WebSocketServer class and send command requests using a WebSockets client. Or you can use a REST API by hosting a HTTP server from the game server with a GDScript HTTP server implementation.

dsnopek commented 1 year ago

In PR #69546, I attempted to implement a small part of this proposal. However, after discussing with @Faless, I've created an alternative proposal that is a little more restricted in scope - see: https://github.com/godotengine/godot-proposals/issues/5970

akien-mga commented 1 year ago

Superseded by #5970.

godotengine / godot-proposals

Implement support for dedicated server export #2756

Describe the project you are working on

Describe the problem or limitation you are having in your project

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

If this enhancement will not be used often, can it be worked around with a few lines of script?

Is there a reason why this should be core and not an add-on in the asset library?

Short FAQ:

991 request is as old as August 2016 (https://github.com/godotengine/godot/issues/6059) and superseded twice -> https://github.com/godotengine/godot/issues/11389 (September 2017 with 46 "thumbs up") -> https://github.com/godotengine/godot-proposals/issues/991 (June 2020)