LiveOverflow / minecraft-hacked

Minecraft:Hacked is a video series exploring various technical areas of Minecraft.
209 stars 12 forks source link

Login timeout when connecting to online services #6

Open robigan opened 2 years ago

robigan commented 2 years ago

Hello there, before diving in. I am making this issue here for 2 reasons. 1) To ask @LiveOverflow's opinion on the matter as he may have already encountered this time out issue and/or help him out with developing this series (For LiveOverflow, if this issue helped you I would really appreciate a shoutout :) and 2) A place to document my findings relating to this issue that shares my same goals. With that out of the way, let me get started.

Ever since the first episode, I was inspired to make a proxy for 2b2t which allows me to host the program on a remote server, use my MC client to connect and initiate a "queue wait" and then disconnect, only to reconnect once I am fairly into the 2b2t queue. I decided to use Quarry as it was easy for me to work with Python.

When I tried to make a simple proxy using episode 1's sample code. I got an issue, I was getting an SSL error. This was simply patched by setting the env variable SSL_CERT_FILE to a location with trust roots setup like so SSL_CERT_FILE="/Users/MY_HOME_FOLDER/homebrew/lib/python3.9/site-packages/certifi/cacert.pem" python3 ./src/proxy.py -b 2b2t.org (See https://stackoverflow.com/questions/33602478/how-to-handle-openssl-ssl-error-while-using-twisted-web-client-agent-on-facebook).

This led me to a second issue, when trying to log into the remote server via the proxy. I would get timed out, I searched this issue up about setting up online server proxying, this led me to the issue https://github.com/barneygale/quarry/issues/135 and the sample code published by @JerryIum but to no avail I got the same issue with the timeout. I decided to turn on more advanced verbose logging by setting logging.basicConfig(level=logging.DEBUG) in my main function. But this wasn't enough, going through Quarry source code. I found that the protocol.py file in the net directory contained the base class for Factories (Factories are functions which initiate classes for you). In which it had a property called log_level which could be set to the enums exposed by logging to turn on more verbose logging. This is the output I finally got when launching my proxy server

DEBUG:MyDownstream{192.168.105.18}:Connection made
DEBUG:MyDownstream{192.168.105.18}:Packet . recv init/handshake
DEBUG:MyDownstream{192.168.105.18}:Packet . recv login/login_start
DEBUG:MyDownstream{192.168.105.18}:Packet # send login/login_encryption_request
DEBUG:MyDownstream{192.168.105.18}:Packet . recv login/login_encryption_response
DEBUG:MyDownstream{192.168.105.18}:Encryption enabled
INFO:MyDownstream{192.168.105.18}:robigan1 has joined.
DEBUG:MyDownstream{192.168.105.18}:Packet # send login/login_set_compression
DEBUG:MyDownstream{192.168.105.18}:Compression threshold set to 256 bytes
DEBUG:MyDownstream{192.168.105.18}:Packet # send login/login_success
DEBUG:MyDownstream{192.168.105.18}:Packet # send play/disconnect
INFO:MyDownstream{192.168.105.18}:Closing connection: Connection timed out
INFO:MyDownstream{192.168.105.18}:robigan1 has left.
DEBUG:MyDownstream{192.168.105.18}:Connection lost

In this log, we can clearly see that there's no logging by the Upstream. So unless the upstream is dying inside with the verbosity I set it, it's basically doing nothing.

Auditing Quarry source code, I found 2 interesting things. When creating a connection using a proxy. The function (in proxy.py) called downstream_ready() is called once the client has connected to the proxy, that function calls it's connect function which causes the upstream proxy to connect. One issue I noted was that in all circumstances, the proxy creates an Offline profile to connect to all upstream servers. This is an issue as this means the connect function needs overwriting. Or specifically, the make_profile function needs overwriting so that we may introduce an Online Profile. Last thing, under normal circumstances, once the upstream has connected. It calls the bridge's upstream_ready function, this function enables forwarding by default which causes "Packet handlers in the Upstream and Downstream cease to be called". This hints to the fact that the Quarry dev has intended for developers to write their own authentication functions on the Upstream and Downstream functions. And once the connection is complete, for the rest of the logic to be handled by the Bridge. Personally, to solve this issue, I intend to grab the access token from the client when connecting to the Downstream and then send that to the Upstream for use with connection to remote servers.

Attached is the current state of my proxy.py

Jerrylum commented 2 years ago

Hmmm, oh yea, I just remember a thing. Could you please start a local server on your computer, then try to connect it? if you can connect to a local server but not a remote server, then here's why:

In fact, did you know when you are trying to connect to a remote minecraft server, for example "minehut.com", you are trying to connect to "_minecraft._tcp.minehut.com" instead?

Here is the implemenation in net.minecraft.client.multiplayer.resolver.ServerRedirectHandler class: image

You see, every time you connect to a Minecraft server, the client tries to resolve an SRV record first. If the client finds something, it uses that address, otherwise use the original one.

The reason why this is a thing is that you can separate your web server and Minecraft server easily. For example, you can browse the website and join the Minecraft server at the same domain name "minehut.com" but they actually have a different IP.

Unfortunately, Quarry doesn't have this feature. As a result, you might want to change your connect address to "_minecraft._tcp.2b2t.org" manually.

For me, I used the library "minecraft-protocol" on Node.js because I like Typescript they fixed the problem. Here is my project:

https://github.com/Jerrylum/2ndDevice

I was also inspired by LiveOverflow to make a mirror proxy. It allows you to control one player on two Minecraft clients. I first wrote my project in python with Quarry but then rewrote it in Typescript Node.js with minecraft-protocol. If you're comfortable with Javascript, you can clone the Proxy I wrote in my project and start working on it.

robigan commented 2 years ago

Huh that would make sense, I forgot about the SRV records lol

robigan commented 2 years ago

Still doesn't change the fact that it's trying to access the remote server using an Offline profile or does that not matter?

robigan commented 2 years ago

Ok so here's some progress, after having talked to with @JerryIum he recommended using the SRV address as the connection, but that's not possible as Quarry doesn't look up SRV records. So I went ahead and ran dig _minecraft._tcp.2b2t.org SRV on my Mac to get the results of the record. This returned connect.2b2t.org, which with an A record lookup returns a CNAME to the proper connect servers.

image

Sweet. So by punching in connect.2b2t.org we're finally connecting to 2b2t. This is a dumb mistake, but it's a mistake with a valuable lesson about how certain aspects of the internet works.

From there we tried it and the error/issue finally changed, so progress was made. The problem that I mentioned above about the proxy using an offline client still persisted, though. You see, the way a Minecraft client and Minecraft server authenticate and know that each other are the real deal is via the session servers. So with great thanks to @Jerrylum where which he explained to me how authentication works. When the client connects, it sends a POST request to Mojang session servers saying it has connected to this server ID, here's my UUID and my access token to verify who I really am. The server then asks Mojang in a GET request if the user with the username it's claiming to connect through has indeed connected to that server ID and if the response code is 204 then the authentication flow is complete. I asked Jerry if it's possible to copy 2b2t's server ID and set that as the proxy server's ID, but he explained you can't as the ID is cryptographically derived from the server's private key. More can be read over at the protocol explanation page LiveOverflow has linked.

image Copyright for this image goes to Jerry Lum

EXTRA INFO FOR DOCUMENTATION PURPOSES

In my case, I want this to be able to be use the proxy as a queuing system for me and my friends. The most likely outcome will be that by using a custom name-spaced packet between the Downstream server and the remote client. I am able to write a mod for the client that receives the name space packet and modifies the request the client makes to Mojang session servers so that Mojang thinks that the client has indeed connected to 2b2t.org (I have ought to check if there's any IP based checking though, unless it'll be the proxy that sends a request to Mojang session servers on behalf of the client with the client providing the Access token, but that's janky and insecure)

3arthquake3 commented 2 years ago

I tried this, it sometimes work sometimes doesn't.. when it doesn't it would say: Can't log into online server while using offline profile..

LiveOverflow commented 2 years ago

Keeping the issue open for easier discovery, because the issue has a really great discussion and information :)

robigan commented 2 years ago

That is fine by me! Jerry and I were happy to help

robigan commented 2 years ago

I tried this, it sometimes work sometimes doesn't.. when it doesn't it would say: Can't log into online server while using offline profile..

Yes that is an issue that I mentioned in my last comment before closing it, it's something that I am trying to patch, but I think I need to get back to @Jerrylum as the library was crashing with HTTP requests when trying to make a Profile that'll let me join online servers. As of now, school is an issue as I have like a bunch of red in my ManageBac dashboard (if you know what that means, good on you cus this shithole of a school management platform makes me go crazy)

robigan commented 2 years ago

I tried this, it sometimes work sometimes doesn't.. when it doesn't it would say: Can't log into online server while using offline profile..

Yes that is an issue that I mentioned in my last comment before closing it, it's something that I am trying to patch, but I think I need to get back to @Jerrylum as the library was crashing with HTTP requests when trying to make a Profile that'll let me join online servers. As of now, school is an issue as I have like a bunch of red in my ManageBac dashboard (if you know what that means, good on you cus this shithole of a school management platform makes me go crazy)

Again for documentation purposes; This is my log when trying to connect:

DEBUG:root:Arguments parsed
INFO:root:Factory created proxying to connect.2b2t.org:25565

INFO:root:Factory listening on 0.0.0.0:25565
Running reactor...
DEBUG:MyDownstream{172.28.14.40}:Connection made
DEBUG:MyDownstream{172.28.14.40}:Packet . recv init/handshake
DEBUG:MyDownstream{172.28.14.40}:Packet . recv status/status_request
DEBUG:MyDownstream{172.28.14.40}:Packet # send status/status_response
DEBUG:MyDownstream{172.28.14.40}:Packet . recv status/status_ping
DEBUG:MyDownstream{172.28.14.40}:Packet # send status/status_pong
DEBUG:MyDownstream{172.28.14.40}:Closing connection
DEBUG:MyDownstream{172.28.14.40}:Connection lost
DEBUG:MyDownstream{172.28.14.40}:Connection made
DEBUG:MyDownstream{172.28.14.40}:Packet . recv init/handshake
DEBUG:MyDownstream{172.28.14.40}:Packet . recv login/login_start
DEBUG:MyDownstream{172.28.14.40}:Packet # send login/login_encryption_request
DEBUG:MyDownstream{172.28.14.40}:Packet . recv login/login_encryption_response
DEBUG:MyDownstream{172.28.14.40}:Encryption enabled
DEBUG:MyDownstream{172.28.14.40}:Connection lost
DEBUG:MyDownstream{172.28.14.40}:Connection made
DEBUG:MyDownstream{172.28.14.40}:Packet . recv init/handshake
DEBUG:MyDownstream{172.28.14.40}:Packet . recv status/status_request
DEBUG:MyDownstream{172.28.14.40}:Packet # send status/status_response
DEBUG:MyDownstream{172.28.14.40}:Packet . recv status/status_ping
DEBUG:MyDownstream{172.28.14.40}:Packet # send status/status_pong
DEBUG:MyDownstream{172.28.14.40}:Closing connection
DEBUG:MyDownstream{172.28.14.40}:Connection lost
DEBUG:MyDownstream{172.28.14.40}:Connection made
DEBUG:MyDownstream{172.28.14.40}:Packet . recv init/handshake
DEBUG:MyDownstream{172.28.14.40}:Packet . recv login/login_start
DEBUG:MyDownstream{172.28.14.40}:Packet # send login/login_encryption_request
DEBUG:MyDownstream{172.28.14.40}:Packet . recv login/login_encryption_response
DEBUG:MyDownstream{172.28.14.40}:Encryption enabled
INFO:MyDownstream{172.28.14.40}:robigan1 has joined.
DEBUG:MyDownstream{172.28.14.40}:Packet # send login/login_set_compression
DEBUG:MyDownstream{172.28.14.40}:Compression threshold set to 256 bytes
DEBUG:MyDownstream{172.28.14.40}:Packet # send login/login_success
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.minecraftservices.com:443
DEBUG:urllib3.connectionpool:https://api.minecraftservices.com:443 "GET /minecraft/profile HTTP/1.1" 200 555
Unhandled error in Deferred:

Traceback (most recent call last):
  File "/Users/MY_HOME_DIR/Documents/Source/2b2t-proxy/venv/lib/python3.9/site-packages/twisted/internet/defer.py", line 857, in _runCallbacks
    current.result = callback(  # type: ignore[misc]
  File "/Users/MY_HOME_DIR/Documents/Source/2b2t-proxy/venv/lib/python3.9/site-packages/quarry/net/http.py", line 43, in _callback2
    d0.callback(json.loads(body.decode('ascii')))
  File "/Users/MY_HOME_DIR/Documents/Source/2b2t-proxy/venv/lib/python3.9/site-packages/twisted/internet/defer.py", line 661, in callback
    self._startRunCallbacks(result)
  File "/Users/MY_HOME_DIR/Documents/Source/2b2t-proxy/venv/lib/python3.9/site-packages/twisted/internet/defer.py", line 763, in _startRunCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File "/Users/MY_HOME_DIR/Documents/Source/2b2t-proxy/venv/lib/python3.9/site-packages/twisted/internet/defer.py", line 857, in _runCallbacks
    current.result = callback(  # type: ignore[misc]
  File "/Users/MY_HOME_DIR/Documents/Source/2b2t-proxy/venv/lib/python3.9/site-packages/quarry/net/server.py", line 116, in auth_ok
    self.player_joined()
  File "/Users/MY_HOME_DIR/Documents/Source/2b2t-proxy/venv/lib/python3.9/site-packages/quarry/net/proxy.py", line 240, in player_joined
    self.bridge.downstream_ready()
  File "/Users/MY_HOME_DIR/Documents/Source/2b2t-proxy/venv/lib/python3.9/site-packages/quarry/net/proxy.py", line 136, in downstream_ready
    self.connect()
  File "/Users/MY_HOME_DIR/Documents/Source/2b2t-proxy/venv/lib/python3.9/site-packages/quarry/net/proxy.py", line 108, in connect
    self.upstream_profile = self.make_profile()
  File "/Users/MY_HOME_DIR/Documents/Source/2b2t-proxy/./src/proxy.py", line 58, in make_profile
    myProfile = Profile.from_token('(skip)', accessToken, myUsername, myUuid)
  File "/Users/MY_HOME_DIR/Documents/Source/2b2t-proxy/venv/lib/python3.9/site-packages/quarry/net/auth.py", line 133, in from_token
    display_name, UUID.from_hex(uuid))
  File "/Users/MY_HOME_DIR/Documents/Source/2b2t-proxy/venv/lib/python3.9/site-packages/quarry/types/uuid.py", line 8, in from_hex
    return cls(hex=hex)
  File "/Users/MY_HOME_DIR/homebrew/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/uuid.py", line 174, in __init__
    hex = hex.replace('urn:', '').replace('uuid:', '')
builtins.AttributeError: 'UUID' object has no attribute 'replace'

^CINFO:MyDownstream{172.28.14.40}:robigan1 has left.
DEBUG:MyDownstream{172.28.14.40}:Connection lost

This seems to be an issue with Quarry's UUID data type, I have yet to still understand if it's me passing the wrong type or smtg to the Profile factory functions, or it's something that needs to be fixed in Quarry. Most likely the issue is with me, but I am not that much of an expert with Python to understand what the correct type would be to be passed, and I haven't seen anything in the docs.

Another interesting thing worth noting is, see how multiple connections are made and dropped at the start of the log? That is my Minecraft client making a status request to the server. These status requests contain the information you'll see when on the server selector in the Multiplayer screen. The thing worthy of note is that the client initializes a new TCP connection EVERY. SINGLE. TIME. a status request is made. In all the time I have played Minecraft, I have noticed that spamming the refresh button isn't rate limited (I might be wrong, but this is my personal experience). Lastly, look at the packets sent, why are there 4 packets transferred (beside the handshake packet) between the client and server. Information only travels 2 ways (one for the request, and one for the response) yet it happens 4 times, better yet, the names of the last 2 packets transferred are called status ping/pong. We might be able to find a DoS exploit here?

Jerrylum commented 2 years ago

Thank you LiveOverflow!

I tried this, it sometimes work sometimes doesn't.. when it doesn't it would say: Can't log into online server while using offline profile..

Btw, Do you still have a problem? @3arthquake3 You can contact me via Jerrylum#0001 on Discord and talk about it.