Allow bots to join friend worlds.

[zLevii]

Bedrock protocol supports joining realms however I assume there is no current support to join friend worlds. It would be great if that could be added.

[rom1504]

@LucienHH could you point to your current WIP work on this? maybe other people could help

[LucienHH]

Yeah sure, here's what I've found so far, joining a peer to peer session is explained here by a support rep:

https://educommunity.minecraft.net/hc/en-us/articles/360047118992-FAQ-IT-Admin-Guide-

Minecraft Education uses a WebRTC signaling service to establish peer-to-peer connections between clients for multiplayer. The establishment of the multiplayer session occurs over web sockets and UDP ports and then the actual peer-to-peer connection occurs over ephemeral ports. Most networks should not need any configuration to support this multiplayer environment but if you do need to configure ports and firewalls, the following information should be helpful:

The signaling connections use wss://signal.franchise.minecraft-services.net The STUN and TURN connections use turn.azure.com / world.relay.skype.com on the 20.202.0.0 / 16 IP range using remote TCP port 443 and remote UDP ports 3478-3481 The peer-to-peer connections between host and joining client use local ephemeral UDP ports specified by the host client (the local port range is defined by the OS) and sent to the joining client via the signaling service

First part to joining a friend's session is by fetching all sessions currently available to join by using the https://sessiondirectory.xboxlive.com API:

POST https://sessiondirectory.xboxlive.com/handles/query?include=relatedInfo,customProperties

Body

{
  "type": "activity",
  "scid":"4fc10100-5f7a-4470-899b-280835760c07",
  "owners": {
    "people": {
      "moniker": "people",
      "monikerXuid": "<xuid>"
    }
  }
}

This will give you a list of sessions to join, within the session's body there will be an object called SupportedConnections which could contain one or more connection types:

{
  "SupportedConnections": [ 
    {
      "ConnectionType": 3,
      "HostIpAddress": "",
      "HostPort": 0,
      "WebRTCNetworkId": 15778080650974306186
    }
  ]
}

Here we have a supported connection of type 3 where WebRTCNetworkId is defined, we'll need this later.

We now need to connect to the signalling channel where we'll send and receive offers/answers. Mojang have a WebSocket connection available at wss://signal.franchise.minecraft-services.net/ws/v1.0/signaling/<20_digit_int>. I haven't been able to decipher where this 20 digit int comes from, potentially randomly generated?

To connect to the WebSocket we first have to get an MCToken, we can do so from the endpoint below, this will most likely need implementing into prismarine-auth:

POST https://authorization.franchise.minecraft-services.net/api/v1.0/session/start

Body

{
  "device": {
    "applicationType": "MinecraftPE",
    "capabilities": ["RayTracing"],
    "gameVersion": "1.20.51",
    "id": "<device_id>",
    "memory": "34185707520",
    "platform": "Windows10",
    "playFabTitleId": "20CA2",
    "storePlatform": "uwp.store",
    "treatmentOverrides": null,
    "type": "Windows10"
  },
  "user": {
    "language": "en",
    "languageCode": "en-GB",
    "regionCode": "GB",
    "token": "<playfab_token>"
    "tokenType": "PlayFab"
  }
}

Response

{
  "result": {
    "authorizationHeader": "<MCTOKEN>",
    "validUntil": "2024-01-14T23:43:10Z",
    "treatments": [
      // bunch of flags
    ],
    "configurations": {
      "minecraft": {
        "id": "Minecraft",
        "parameters": {
          // bunch of key values
        }
      }
    }
  }
}

With the MCToken we can now create a connection to the WebSocket service. The server will immediately respond with ICE server credentials upon connecting to the WebSocket.

Client <-- Server

{
  "Type": 2,
  "From": "Server",
  "Message": "{\"Username\":\"\",\"Password\":\"\",\"ExpirationInSeconds\":172799,\"TurnAuthServers\":[{\"Username\":\"\",\"Password\":\"\",\"Urls\":[\"stun:relay.communication.microsoft.com:3478\",\"turn:relay.communication.microsoft.com:3478\"]}]}"
}

The client then responds with multiple requests which are documented below

NOTE: The To propety is the WebRTCNetworkID we saved from the session

Client --> Server

{
  "Message": "CONNECTREQUEST 9806856835729287287 v=0\r\no=- 5487540117316254753 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0\r\na=extmap-allow-mixed\r\na=msid-semantic: WMS\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:GTsq\r\na=ice-pwd:<pass>\r\na=ice-options:trickle\r\na=fingerprint:<fingerprint>\r\na=setup:actpass\r\na=mid:0\r\na=sctp-port:5000\r\na=max-message-size:262144\r\n",
  "To": 15778080650974306186,
  "Type": 1
}

Client --> Server

{
  "Message": "CANDIDATEADD 9806856835729287287 candidate:4152947846 1 udp 2122265344 <IPV6> 49370 typ host generation 0 ufrag GTsq network-id 4 network-cost 10",
  "To": 15778080650974306186,
  "Type": 1
}

Client --> Server

{
  "Message": "CANDIDATEADD 9806856835729287287 candidate:3061656305 1 udp 2122199808 <IPV6> 49371 typ host generation 0 ufrag GTsq network-id 5 network-cost 10",
  "To": 15778080650974306186,
  "Type": 1
}

Client --> Server

{
  "Message": "CANDIDATEADD 9806856835729287287 candidate:2426374300 1 udp 2122134272 <IPV6> 49372 typ host generation 0 ufrag GTsq network-id 8 network-cost 10",
  "To": 15778080650974306186,
  "Type": 1
}

Client --> Server

20.202.1.9 is the STUN/TURN server

{
"Message": "CANDIDATEADD 9806856835729287287 candidate:737026903 1 udp 41558015 20.202.1.9 52312 typ relay raddr <public-IPV4> rport 49375 generation 0 ufrag GTsq network-id 1 network-cost 10",
"To": 15778080650974306186,
"Type": 1
}

After this we then receive the response from the server and 5 other CANNIDATEADD responses, I've only documented the first response and the first CANNIDATEADD response below:

Client <-- Server

{
  "Type": 1,
  "From": "15778080650974306186",
  "Message": "CONNECTRESPONSE 9806856835729287287 v=0\r\no=- 6021002969452554251 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0\r\na=extmap-allow-mixed\r\na=msid-semantic: WMS\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:ji75\r\na=ice-pwd:<pass>\r\na=ice-options:trickle\r\na=fingerprint:<fingerprint>\r\na=setup:active\r\na=mid:0\r\na=sctp-port:5000\r\na=max-message-size:262144\r\n"
}

Client <-- Server

{
  "Type": 1,
  "From": "15778080650974306186",
  "Message": "CANDIDATEADD 9806856835729287287 candidate:148451967 1 udp 2122129151 <local_world_ipv4> 51125 typ host generation 0 ufrag ji75 network-id 1 network-cost 10"
}

In the above response was the local IP for the world that was hosted on my mobile which allowed me to connect to the sesssion. I haven't been able to see what joining a session hosted outside of my local network would look like but I imagine it'd be a fairly similar process. The part that needs working on is the connection within the signalling channel, sending the correct SDP request to the host to receive the public IP. I haven't been able to spend much more time on this but if anyone's well experienced with this protocol please chime in.

[extremeheat]

Thanks for the write up. I don't have experience with WebRTC, so I can't comment much on this protocol. Do you think there is a difference in implementation here as opposed to other WebRTC peer-to-peer application protocols? I'd bet that they did not invent anything here, more likely it seems they are using some existing libraries to facilitate the communication, https://github.com/zenomt/rtmfp-cpp/tree/main seems interesting and maybe related.

As for the 20 digit integer: if you can, route the domain to a local WebSocket server that acts as a proxy and try erasing this field when forwarding request to the remote server and check if the peer to peer connection still works or not. That will give an idea on the significance/randomness of the ID.

As for the session description Message's, this should be normal WebRTC comms, no? It should be possible to either use some lib to handle the protocol here or just replay vanilla messages with small modifications as necessary.

[LucienHH]

No worries, thanks for jumping in on this. I'll have a look at figuring out the 20 digit int later tonight, upon looking at Mojangs telemetry they call this value WorldId I'm not sure where they're getting this from but if that helps jog a memory there you go. In terms of the SDP crap, yes it's following normal protocol afaik.

In this scuffed POC we connect to the signalling channel, TURN/STUN servers and send the data between. I haven't been able to get this in a polished state yet.

const WebSocket = require('ws');
const { RTCPeerConnection } = require('werift');
const { Authflow, Titles } = require('prismarine-auth')

// webrtcid
// 5696196935114111266

// ws
// https://signal.franchise.minecraft-services.net/ws/v1.0/signaling/15859775084651201335

//https://authorization.franchise.minecraft-services.net/api/v1.0/session/start
// post {"device":{"applicationType":"MinecraftPE","capabilities":["RayTracing"],"gameVersion":"1.20.51","id":"c1681ad3-415e-30cd-abd3-3b8f51e771d1","memory":"34185707520","platform":"Windows10","playFabTitleId":"20CA2","storePlatform":"uwp.store","treatmentOverrides":null,"type":"Windows10"},"user":{"language":"en","languageCode":"en-GB","regionCode":"GB","token":"","tokenType":"PlayFab"}}

const main = async () => {

  // WORLDID 16920563543604857349
  const ws = new WebSocket('wss://signal.franchise.minecraft-services.net/ws/v1.0/signaling/5301802620021606730', {
    headers: { Authorization: `MCToken <token>` }
  });

  let resolve

  const promise = new Promise((res) => {
    resolve = res
  }) 

  ws.on('message', (data) => {
    // Potentially need to set remoteDescription here
    const message = JSON.parse(String(data));

    console.log(message)

    const payload = JSON.parse(message.Message);

    resolve(payload);
  })

  const data = await promise

  console.log(data)

  const peer = new RTCPeerConnection({
    iceServers: [
      {
        urls: 'stun:relay.communication.microsoft.com:3478',
        credential: data.Password,
        username: data.Username
      },
      {
        urls: 'turn:relay.communication.microsoft.com:3478',
        credential: data.Password,
        username: data.Username
      },
    ],
    iceTransportPolicy: "all"
  })

  const dc = peer.createDataChannel('test')

  peer.oniceconnectionstatechange = () => {
    console.log(
      "oniceconnectionstatechange",
      peer.iceConnectionState
    );
  };

  peer.onicecandidate = function (evt) {
    console.log(evt)
    if (evt.candidate) {
      // Send the candidate to the other party via signaling channel

      const message = `{
        "Type": 1,
        "To": 15341643931879376243,
        "Message": "CANDIDATEADD 9806856835729287287 ${evt.candidate.candidate}"
      }`

      console.log(message)

      ws.send(message)
    }
  };

  const offer = await peer.createOffer()

  console.log(offer)

  const message = `{
    "Type": 1,
    "To": 15341643931879376243,
    "Message": "CONNECTREQUEST 9806856835729287287 ${offer.sdp}"
  }`

  console.log(message)

  ws.send(message)

  await peer.setLocalDescription(offer)

}

main()

[extremeheat]

If it's a consistent number, then it's either retrieved from the network or stored somewhere in the file system. In case of latter I'd think it could be a hash of the world ID. So not really retrievable, but doesn't sound important either. As for the implementation, you can ask ChatGPT to write some simple back and forth protocol code based on that

[zLevii]

So I just wanted to ask on how POST https://sessiondirectory.xboxlive.com/handles/query?include=relatedInfo,customProperties works.

I've tried getting the data off it however the ip and port are empty strings just the same way as your response is however another friend has managed to directly get the IP and port off that request itself?

What is the signaling and everything with the WS for? I'm a little lost but is that authentication? I suppose you are sending a message to the WS asking to connect and it authorizes that request and allows you to join accordingly and it wont be possible to just slap an IP & Port and be able to join?

[LucienHH]

@zLevii It depends on the session you're connecting to, if the session is a server then it will contain the IP/port and connect using that, however if the connection is P2P then it will need to establish a connection via the signalling channel. It's briefly explained in my initial comment.

[zLevii]

Yeah it's P2P and it makes sense as to why I cannot connect with a direct IP/port. I would have to tell xbox to add me etc.

What if I invite the account to the world? Can I skip the signaling etc?

[LucienHH]

Inviting a player to the session adds them, allowing them to get the 'WebRTCNetworkId' from the connection details. You'd still need to establish a connection with the host via WebRTC.

[LucienHH]

In the below example is a working script which establishes a connection with a peer via Minecraft's signalling channel and successfully returns the hosts connection details.

https://gist.github.com/LucienHH/dab431394dabc38026ee1dd81cd0cdbc

This returns 2 IPv6 addresses which the client then uses to connect to the remote peer, using DTLS v1.2.

From signalling channel:

Packets from wireshark

I'm not sure that bedrock-protocol supports IPv6 and if that we'd need to maintain the connection through WebRTC?

[zLevii]

Inviting a player to the session adds them, allowing them to get the 'WebRTCNetworkId' from the connection details. You'd still need to establish a connection with the host via WebRTC.

Makes sense.

[zLevii]

In the below example is a working script which establishes a connection with a peer via Minecraft's signalling channel and successfully returns the hosts connection details.

https://gist.github.com/LucienHH/dab431394dabc38026ee1dd81cd0cdbc

This returns 2 IPv6 addresses which the client then uses to connect to the remote peer, using DTLS v1.2.

From signalling channel:

Packets from wireshark

I'm not sure that bedrock-protocol supports IPv6 and if that we'd need to maintain the connection through WebRTC?

Interesting!

I assume the gist you linked is everything after getting the WebRTC isn't it?

I'll follow through that example and see how I get along.

@rom1504 Is this something bedrock protocol could support?

[rom1504]

Why not

[zLevii]

Why not

For sure is possible to however we would need your help to chip in as this stuff is kind of beyond my understanding so it would be nice if the team could work alongside @LucienHH to implement this

[LucienHH]

In the below example is a working script which establishes a connection with a peer via Minecraft's signalling channel and successfully returns the hosts connection details. go to...

@extremeheat any thoughts on the protocol change when it's a peer to peer session, bit out of my realm of knowledge?

[extremeheat]

Is all the normal minecraft game traffic over that DTLS protocol? Or is that just an intermediate step? Would need more information on that.

[LucienHH]

All of the peer-to-peer session is over DTLS compared to connecting to a Realm / external server that's over Raknet

[JustTalDevelops]

I've written docs and done various tests with this a couple of months back. More info: https://github.com/df-mc/nethernet-spec https://github.com/df-mc/nethernet-playground