search

fishfolk / bones

An easy-to-use game engine for making real games.
https://fishfolk.org/development/bones/introduction/
Other
210 stars 20 forks source link

feat: implement p2p based networking using `iroh-net` #404

Closed dignifiedquire closed 3 months ago

dignifiedquire commented 3 months ago

This the start to run the networking connections over iroh, allowing direct connections between players.

General Architecture

Testing it out

There is a branch of jumpy using this, which can be used to try it out https://github.com/dignifiedquire/jumpy/tree/feat-iroh-networking

Matchmaker Servers

Work to be done

MaxCWhitehead commented 3 months ago

Yeah this is great work - excited to play with it more. Once we get this in bones can deploy another matchmaker and switch jumpy over and see how things go.

MaxCWhitehead commented 3 months ago

I tried testing this with a docker imagine running on my same machine as clients, and found there is high latency and spam of this:

2024-05-16T03:58:55.046961Z  INFO poll_send{me=juiy6aqwnxrhan5j}:get_send_addrs{node=g3vsalpcnyeyqk6m}: iroh_net::magicsock::node_map::node_state: new connection type typ=mixed

I am curious how publishing the IP of server works when trying to connect to node ID from inside the LAN (or in my case on the same host) - it seemed to connect, but did not seem to have a reliable / direct UDP connection. (It seems possible there is something we have to change to handle this better - just wanted to note as while I would normally specify the localhost or LAN address to do this, I am no longer sure how to do so with node id.

anywho haven't debugged this much but going to throw this up in a more legit configuration on a VPS in same region as another non-iroh matchmaker to compare with.

EDIT: I think I missed the fact we are on p2p now and not just relaying through matchmaker with iroh connections - testing with a remote server with two local clients I still get same logs / not great ping (seems that using relay https://use1-1.relay.iroh.network which I am 85ms away from).

But the logs show my two clients are sending to each other (our server is not MITM anymore). So issue is not related to location of the matchmaking server, but is impacted by relay as not able to negotiate direct connections on same host. Will have to test with two clients on separate hosts in same lan next.

MaxCWhitehead commented 3 months ago

In case it's helpful for anyone got a tiny server running matchmaker (as of commit e075fc6a, in Texas, USA) can use for testing.

node id: 37zobpkk3vyheqonfph55vh5p5w5wn2ipx6qixcffkfox4gdzpsq

dignifiedquire commented 3 months ago

But the logs show my two clients are sending to each other (our server is not MITM anymore). So issue is not related to location of the matchmaking server, but is impacted by relay as not able to negotiate direct connections on same host. Will have to test with two clients on separate hosts in same lan next.

Yes, that is what I was suspecting the issue to be as well, especially the log line including mixed connection type, means the system is not able to determine if the direct connections are fully reliable.

Running on my machine locally (two games, same host) things worked out well, it might be we are having some trouble detecting appropriate IP addresses due to docker isolations.

dignifiedquire commented 3 months ago

My only minor thought is that before merging we should try to figure out which tokio features we need and remove the full feature.

Yes, fully agreed, I use this to just say give me everything during development and then remove the pieces that I don't end up needing in the end.

Arqu commented 3 months ago

I'm just going to sling a couple wild guesses, maybe they lead somewhere. 1) Is the docker running the network in host or bridge mode for the image. Bridge might interfere here as you basically get a docker "subnet" which then gives bogus IP addresses internally. Technically you need to have a local quasi stun so that it gives the right IP addr of the machine not the container. 2) Port matching might be at play, dunno if it's possible to just expose all ports in bridge mode

Ideally the next thing that gets tested is to try running in HOST mode which binds to your local network interface/stack. That should hopefully resolve both and give us a better understanding of the issue.

MaxCWhitehead commented 3 months ago

I'll try adjusting the container's network mode on local / remote deployments and see if that helps. I'll also test matchmaker running locally without a container. (Problem doesn't have to block merge, just sorting it out)

MaxCWhitehead commented 3 months ago

Updating the remote mm server container to run with host networking (docker run -p 8943:8943/udp --network host matchmaker-iroh) did not resolve the issue with two clients on same host. It is up at 37zobpkk3vyheqonfph55vh5p5w5wn2ipx6qixcffkfox4gdzpsq in case anyone wants to test the configuration - but fairly certain not a docker issue.

I am running another mm server on VPS in same region natively (cargo run) instead of in container at 7v77r7qofyrz4ms4y6offrfji377l5qqatxjqa34z57tgcv7fpkq, I get the same issue. So likely an issue with my local network configuration.

Will do some more tests and look into it further as I have the time. (Next test is two clients separate hosts under same LAN, then can figure out if should be looking host configuration or router / NAT).

MaxCWhitehead commented 3 months ago

@dignifiedquire re:

browser, will likely have to be in some form of disabling this when running in wasm (for now)

Not sure if you are tracking this but we do not support multiplayer in wasm, basically just a demo atm. So disabling all networking in wasm is sufficient resolution, just need to make sure wasm builds.