search

godotengine / godot

Godot Engine – Multi-platform 2D and 3D game engine
https://godotengine.org
MIT License
91.47k stars 21.27k forks source link

[TRACKER] Networking improvements #13947

Closed mhilbrunner closed 4 years ago

mhilbrunner commented 6 years ago

This issue is to collect and discuss improvements to the networking in Godot 3.

Calinou commented 5 years ago

I think @Ploppy3 meant UPnP doesn't work everywhere (e.g. on mobile networks), so it cannot be relied upon in all cases if you want players to be able to host servers.

That said, I think it's a big improvement over not having anything of the sort :smiley:

Faless commented 5 years ago

Oh, I see, well UPNP only makes sense on local networks. What problem are you trying to solve? Peer discovery? In that case you need a discovery service of some kind hosted on a public server. For peer discovery on the internet you need to host your own discovery service, NAT punch-through? That depends on the PacketPeer you use, There was some work for ENet by another contributor in #16034 then merged in #18666 . But in general that's a hard problem to solve and it's unlikely to be 100% reliable (and you still need your own discovery service publicly available on the internet).

Ploppy3 commented 5 years ago

UPNP is known to be unreliable, it won't work for everybody out of the box since some router don't support it or comes with it disabled by default.

Also, I asked for an alternative since I have a problem which seems to be related to the UPNP library used in Godot https://github.com/godotengine/godot/issues/25008

I'm kind of lost at this point and don't know what is the right approach to let someone host a game on Godot without having to do the port-forwarding himself.

NB: I'm new to network

Faless commented 5 years ago

it won't work for everybody out of the box since some router don't support it or comes with it disabled by default.

Of course, because that was finally acknowledged as a security issue and now most routers come with it disabled.

I'm kind of lost at this point and don't know what is the right approach to let someone host a game on Godot without having to do the port-forwarding himself.

You can't with 100% reliability, as you can't with any other piece of software, because there will always be some case where the router/network configuration does not allow it.

Faless commented 5 years ago

You can't with 100% reliability

That is basically why every game I know of tells you which port you need to open to host a match, so if you don't have working UPNP IGD you can manually open them.

Ploppy3 commented 5 years ago

@Faless I understand that, though Warframe and Supreme Commander FAF both use UPNP and work without it. I'm asking for alternatives, what are these games solution to this problem?

I see SupComFAF is talking about proxy but why would a server be magically allowed access on your router? I'm quite confused. (see the 'magic' part, that's where I'm lost ;) )

Faless commented 5 years ago

I'm asking for alternatives, what are these games solution to this problem?

Use UPnP, if that fails, (optionally try NAT punch-through, if that fails) ask the user to manually open ports. SupComFAF in fact has a The longer but definitive fix section where it asks the user to manually open ports

trooper commented 5 years ago

Docs for movement interpolation & reconciliation

I see there's a sub-issue opened for adding documentation / tutorials around interpolation/reconciliation, but no issue opened around adding any engine-level support for this. Was this concluded to be a non-goal for the engine (and if so, why)?

Regarding authority - it seems that the current approach is relying on clients to compute the physics and send the results to the server. I still dont fully understand how that would work and how potential conflicts between what multiple clients computed would be resolved. Furthermore, it seems very prone to cheaters / compromised clients.

Is there an issue # to track adding support for relying fully on server to simulate all the physics and having client-side prediction?

Thanks!

Faless commented 5 years ago

but no issue opened around adding any engine-level support for this. Was this concluded to be a non-goal for the engine (and if so, why)?

I guess the discussion is still ongoing, the question is mainly if this can be done in a general way that is useful in common cases. What I've encountered with my RT game is that some interpolations depends on game mechanics (e.g. custom integration), so it's kinda hard to generalize.

The way I do it is to set, in the clients, all the physics objects to static, disable the collisions, attach a Tween node that interpolate position/velocity/acceleration (sent by the server) but also apply the custom_integration function. The clients are then only allowed to send their inputs.

Works very smoothly in my case, but I'm not sure if this could be good as a general approach.

I do not like reconciliation, reconciliation makes the game worse for people on good connections, and it might even help cheaters. It's something I try to avoid unless it's absolutely necessary. I've been saying this for a while, and gladly I've recently found out I'm not the only one: https://youtu.be/vPjpiokAUas?t=221

Regarding authority [...] it seems very prone to cheaters / compromised clients.

That depends on the way you implement your network code.

The multiplayer example in the demos is non-authoritative (i.e. peers send their own positions/calls and the others accepts them blindly).

You can do an authoritative server by e.g. having the players only send input state to the server/host which applies them to the player nodes (and is master on all the other nodes too)

You can, if you want, implement other techniques like consensus algorithms, turn-based strategies like in some RTS networking, etc.

The decision is left to the game dev, as there's no fit for all solution.

We'll be discussing networking/multiplayer improvements at the upcoming Godot events, and we'll update the roadmap in regard.

As much as we understand the wishes for an easy-to-use system for some common networking cases, generalizing them is not an easy task, and sadly not always possible.

raymoo commented 5 years ago

19110 talks about some potential man in the middle attacks, but it occurred to me that Godot multiplayer may also be vulnerable to spoofed RPCs (a peer sends an RPC that looks like it came from another peer). This is a more pressing issue because then an attacker doesn't need any special power other than knowing another user's IP address, and then they could do things like disconnect the other player or make them send in-game resources to the attacker. Knowing someone's IP address is plausible since people may both host and join games, and they need to publish their IP address to host.

Protection against this kind of attack wouldn't need a full DTLS implementation (though I think a DTLS implementation would also protect against this). I think it would be enough to include an HMAC of each RPC, using a key (unique for each client) sent from the server to the client in plaintext at the beginning of the connection. Obviously this wouldn't protect against MITM attacks, but an attacker wouldn't be able to forge packets unless they can eavesdrop on the connection.

Faless commented 5 years ago

but it occurred to me that Godot multiplayer may also be vulnerable to spoofed RPCs (a peer sends an RPC that looks like it came from another peer).

There are checks for that.

This is a more pressing issue because then an attacker doesn't need any special power other than knowing another user's IP address.

And you need to be able to spoof your IP address which every legit ISP will not allow you to do, plus ENet has it's own extra session id, and you'll need to know a bunch of other parameters (like path caches, sequence numbers, etc).

I don't think implementing hmac would make sense under those circumstances.

raymoo commented 5 years ago

This stack exchange post (https://security.stackexchange.com/questions/1062/why-dont-isps-filter-on-source-address-to-prevent-spoofing) suggests ISPs typically didn't filter spoofed packets in 2010. Do you have a source reporting which ISPs currently filter spoofed packets?

The session ID in ENet is only 8 bits and the sequence number is 16 bits, so those could eventually be guessed. I am not sure whether they are initialized randomly or not, but if not, they could be guessed even more reliably. Do you happen to know if there is an official specification of ENet's protocol? I had trouble answering these questions to myself since I couldn't find one.

I don't really understand Godot's plumbing that well, so I couldn't say whether path caches would be sufficiently unguessable.

Faless commented 5 years ago

As stated in a more recent answer to that question an increasing number of ISP have realized the impact of ip spoofing in terms of costs, and are thus implementing anti-spoofing techniques.

Additionally, knowing the IP of the server is not really enough, you would also need the IP of the client. Not only that, you would also need to know the randomly assigned UDP port of the client (if we are talking about enet).

That said, if there's any proof of concept that works by just knowing the server IP/port let me know.

About enet, the only documentation I know of is here: http://enet.bespin.org/ , the rest I gathered from the source code.

On Wed, Jun 5, 2019, 18:48 raymoo notifications@github.com wrote:

This stack exchange post ( https://security.stackexchange.com/questions/1062/why-dont-isps-filter-on-source-address-to-prevent-spoofing) suggests ISPs typically didn't filter spoofed packets in 2010. Do you have a source reporting which ISPs currently spoof packets?

The session ID in ENet is only 8 bits and the sequence number is 16 bits, so those could eventually be guessed. I am not sure whether they are initialized randomly or not, but if not, they could be guessed even more reliably. Do you happen to know if there is an official specification of ENet's protocol? I had trouble answering these questions to myself since I couldn't find one.

I don't really understand Godot's plumbing that well, so I couldn't say whether path caches would be sufficiently unguessable.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/13947?email_source=notifications&email_token=AAM4C3RZXHJE5D64PMKFPBTPY7U6TA5CNFSM4EGQQ3K2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXAKMPA#issuecomment-499164732, or mute the thread https://github.com/notifications/unsubscribe-auth/AAM4C3T77WQAVNDT4BMMHILPY7U6TANCNFSM4EGQQ3KQ .

raymoo commented 5 years ago

As stated in a more recent answer to that question an increasing number of ISP have realized the impact of ip spoofing in terms of costs, and are thus implementing anti-spoofing techniques.

An increasing number is not the same as "every legit ISP". Without specific figures it could mean 10% were before, and 25% are now.

Additionally, knowing the IP of the server is not really enough, you would also need the IP of the client.

When I said an attacker knows the IP address of another user, I meant another client. In the same post I give a reason why it's plausible that an unprivileged attacker would know the IP address of another player.

Not only that, you would also need to know the randomly assigned UDP port of the client (if we are talking about enet).

The port number is 16-bit, which could be brute forced. However, the combination of port number + sequence number would be more difficult to brute force if both the port number and initial sequence number are assigned randomly. From Wikipedia I know that Linux has a port number range of around 16 bits, and Windows around 15 bits. I think on at least Linux it is assigned in some random way and probably can't be predicted easily unless the attacker can observe the other client's connection status. I don't know if ENet initializes its initial sequence number randomly, but if it does then I think the attacks I was worried about would take too long to be useful in a game.

About enet, the only documentation I know of is here: http://enet.bespin.org/ , the rest I gathered from the source code.

I was hoping there was a specification of the protocol independent from the implementation, but thank you for checking.

Faless commented 5 years ago

An increasing number is not the same as "every legit ISP". Without specific figures it could mean 10% were before, and 25% are now.

A non-complete list: https://www.manrs.org/isps/participants/ Feel free to go over it, don't expect ISP names there, but bigger consortiums, e.g. for Italy MiX includes all ISPs I know and some I've never heard of.

When I said an attacker knows the IP address of another user, I meant another client. In the same post I give a reason why it's plausible that an unprivileged attacker would know the IP address of another player.

You mean the host+join scenario? Well, in that case, the join part is most likely on loopback, which is pretty hard to spoof.

I don't know if ENet initializes its initial sequence number randomly, but if it does then I think the attacks I was worried about would take too long to be useful in a game.

Now that I better think of it, sequence number is only added to reliable and unreliable-ordered packets, so we can't really count on that.

I would still prefer to be sure that this is actually exploitable in a real case scenario before adding more code, but in any case, if we are going to implement it, it should be limited to the ENet peer, not affecting the whole MultiplayerAPI.

raymoo commented 5 years ago

A non-complete list: https://www.manrs.org/isps/participants/ Feel free to go over it, don't expect ISP names there, but bigger consortiums, e.g. for Italy MiX includes all ISPs I know and some I've never heard of.

Thanks. This doesn't really give a proportion, but I do recognize some large US ISPs when I searched.

You mean the host+join scenario? Well, in that case, the join part is most likely on loopback, which is pretty hard to spoof.

I mean where a player sometimes hosts a game for other people to join, and sometimes joins other people's games. If they advertise their games in (for example) a Discord server, other players may know their IP address from previous interactions.

I would still prefer to be sure that this is actually exploitable in a real case scenario before adding more code, but in any case, if we are going to implement it, it should be limited to the ENet peer, not affecting the whole MultiplayerAPI.

Does this mean it wouldn't affect RPC calls? Or do you just mean that the code changes should be localized to the ENet peer?

Faless commented 5 years ago

I mean where a player sometimes hosts a game for other people to join, and sometimes joins other people's games. If they advertise their games in (for example) a Discord server, other players may know their IP address from previous interactions.

It seems to me it's a case that requires some quite special conditions and a lot of effort from the attacker, with limited chance of success, and little to gain. I wouldn't worry too much about it, as we are talking about games here, not mission critical systems.

Does this mean it wouldn't affect RPC calls? Or do you just mean that the code changes should be localized to the ENet peer?

That it should be localized to the ENet peer and made optional. Other multiplayer peers (websocket and upcoming webrtc) don't need it, making that a waste of precious bytes and computational power.

raymoo commented 5 years ago

It seems to me it's a case that requires some quite special conditions

It's a pretty common situation for some types of games, such as fighting games without matchmaking, where people typically post their IP address and invite people to join.

and a lot of effort from the attacker

It only requires the effort once to be reused by the attacker, and maybe other attackers who obtain the exploiting program from the original attacker.

with limited chance of success,

If there's a command that doesn't require a sequence number, and is a valid command at any point in the game (such as a client notifying disconnection), then it could be (I am guessing) exploited reliably.

and little to gain.

I don't think this is true, at least for moderately-successful games. Some potential rewards (using the example of disconnecting a player):

That it should be localized to the ENet peer and made optional. Other multiplayer peers (websocket and upcoming webrtc) don't need it, making that a waste of precious bytes and computational power.

Sounds good to me.

Faless commented 5 years ago

If there's a command that doesn't require a sequence number, and is a valid command at any point in the game (such as a client notifying disconnection), then it could be (I am guessing) exploited reliably.

Well, I tried to explain you why this is not a trivial thing to do (filtering, sequencing, session IDs, ecc), but since you prefer making assumption and guesses that it's easy to do, there's clearly no point in arguing, as you clearly made your final decision already.

Sounds good to me.

While I still don't think it's going to be much useful, feel free to open a PR, I'll make sure to review it.

raymoo commented 5 years ago

(filtering, sequencing, session IDs, ecc), but since you prefer making assumption and guesses that it's easy to do, there's clearly no point in arguing, as you clearly made your final decision already.

I thought you said that not all packets used by Godot had sequence numbers? And session IDs are only a byte and probably not random.

Faless commented 5 years ago

But you assumed that no sequence was used for "disconnect" messages, and you are now assuming that sessionId is not random

On Tue, Jun 11, 2019, 15:50 raymoo notifications@github.com wrote:

(filtering, sequencing, session IDs, ecc), but since you prefer making assumption and guesses that it's easy to do, there's clearly no point in arguing, as you clearly made your final decision already.

I thought you said that not all packets used by Godot had sequence numbers? And session IDs are only a byte and probably not random.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/13947?email_source=notifications&email_token=AAM4C3V66PNWILGEL2WNOO3PZ63SXA5CNFSM4EGQQ3K2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXNMJII#issuecomment-500876449, or mute the thread https://github.com/notifications/unsubscribe-auth/AAM4C3S45SQI776365B75ALPZ63SXANCNFSM4EGQQ3KQ .

organicpencil commented 5 years ago

It'd be cool to have an encryption switch for the high-level API which would solve 2 problems at once. But I'm not using it, so don't expect a PR from me :]

Faless commented 5 years ago

It'd be cool to have an encryption switch for the high-level API which would solve 2 problems at once. But I'm not using it, so don't expect a PR from me :]

Yeah, encryption would be much more useful for a general case, and I'm planning to investigate that (again, at ENet level) once I implement DTLS support in Godot.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/13947?email_source=notifications&email_token=AAM4C3UTTGDKJYCSKWS3JQLPZ7HAXA5CNFSM4EGQQ3K2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXNWYTI#issuecomment-500919373, or mute the thread https://github.com/notifications/unsubscribe-auth/AAM4C3SJFCWOGCAZNNAAN43PZ7HAXANCNFSM4EGQQ3KQ .

raymoo commented 5 years ago

But you assumed that no sequence was used for "disconnect" messages

ENet itself allows receiving the ENet disconnect packet as unsequenced. It looks like it has some sort of check (https://github.com/lsalzman/enet/blob/0eaf48eeb0d94a18d079378d8b76d588832ce838/protocol.c#L478) in unsequenced packets, but seems like it would accept a large range of values, so is probably meant for some other purpose than security. In particular it looks like if the command had unsequencedGroup set to 0, it would always be accepted (I could be wrong).

you are now assuming that sessionId is not random

I looked at the code for determining session IDs (https://github.com/lsalzman/enet/blob/0eaf48eeb0d94a18d079378d8b76d588832ce838/protocol.c#L340, https://github.com/lsalzman/enet/blob/0eaf48eeb0d94a18d079378d8b76d588832ce838/protocol.c#L340) and I believe it starts at zero for each peer and counts up with extra connections. I'm not sure, but in any case it looks pretty deterministic.

nxrighthere commented 5 years ago

The only realistic and well-known vulnerability in the ENet is the Blind Masquerade, which is explained in RFC 4960. We solved this by using four-way handshake and cookies. A similar attempt to solve this was done in this fork and further commits there, but no idea if his implementation works in practice.

For everything else we use encryption based on the Noise protocol.

Faless commented 5 years ago

In case someone is interested, this service provide spoofing tests and stats: https://spoofer.caida.org/summary.php

raymoo commented 5 years ago

These graphs measure how many client addresses are capable of spoofing as a different source address. The portion is low but significant (a bit smaller than a quarter), so an attacker whose ISP performs filtering can look for a VPS in a region or behind an ISP that doesn't filter source addresses. Thanks for finding these statistics.

I referenced the abstract for the paper for that project for this post: https://www.usenix.org/legacy/events/sruti05/tech/beverly.html

Razzeeyy commented 5 years ago

Ability to kick/disconnect peers as the server

I believe this should be in an API for ~MultiplayerAPI~ NetworkedMultiplayerPeer not the specific peer implementations. Because it really feels inconsistent and awkward to cast to specific peer implementation, since the rest of the API pretty much would expect this kind of functionality to be present.

Jummit commented 4 years ago

The second issue was closed(Method to get networking stats): https://github.com/godotengine/godot/issues/8763#issuecomment-577178610

Calinou commented 4 years ago

Note: For any items not having a proposal open yet, please open one proposal per feature (don't lump features together).

Feature and improvement proposals for the Godot Engine are now being discussed and reviewed in a dedicated Godot Improvement Proposals (GIP) (godotengine/godot-proposals) issue tracker. The GIP tracker has a detailed issue template designed so that proposals include all the relevant information to start a productive discussion and help the community assess the validity of the proposal for the engine.

The main (godotengine/godot) tracker is now solely dedicated to bug reports and Pull Requests, enabling contributors to have a better focus on bug fixing work. Therefore, we are now closing all older feature proposals on the main issue tracker.

If you are interested in this feature proposal, please open a new proposal on the GIP tracker following the given issue template (after checking that it doesn't exist already). Be sure to reference this closed issue if it includes any relevant discussion (which you are also encouraged to summarize in the new proposal). Thanks in advance!