Important announcement that may affect your project

[luciopaiva]

Hey folks,

I'm a backend engineer at Zwift and want to bring to your attention some upcoming changes that are likely to affect your project.

In July we are rolling out a new encryption protocol that will affect all UDP and TCP communication between client and server, and also between client and the Zwift Companion app (ZC), in order to meet a new requirement from Google Play (they recently changed their user data safety policy).

The rollout will span several days and you may see both encrypted and unencrypted clients running concurrently during that period. It is expected that by July 20th we will have all our communication fully encrypted.

After July 20th, unencrypted clients will be considered deprecated, but still supported. Eventually, however, Zwift will enforce encryption for all clients, but that date is still to be determined.

How exactly that affects your project will depend on what it does:

• server clones: they won't be affected by encryption while unencrypted communication is still supported. Encrypted clients must negotiate encryption with the server, and they will simply fall back to unencrypted communication if the server doesn't know any better. When encryption is enforced, however, these projects will stop working;

• ZC clones: they won't be affected during that initial phase either. Encrypted clients must also negotiate encryption with ZC, and they will also fall back to unencrypted in case of pre-encryption ZCs. When encryption is enforced, however, these projects will stop working;

• client sniffers: those will be immediately impacted when rollout starts. Not only is the protocol specification changing, but the very payload being sniffed by those projects is now encrypted. Logic that relies on reading user data from network packets will simply fail if the data is encrypted, as one would expect;

• BLE and ANT+ devices: projects that emulate either BLE or ANT+ devices won't be affected, not even after encryption is enforced (unless of course they also fit any of the roles aforementioned). We are not changing those communication protocols in any way.

Please note that this change is required to secure user data, not to discourage community projects. We are internally discussing if and how we can help community projects go through the change and continue working. That said, please have in mind that this is not merely an engineering decision, but it must also involve other departments like Product and Legal. We will post further updates here as soon as we have any news.

We'll also try our best to answer questions, depending on the availability of our backend engineering team. You can trust replies from any of the engineers listed here.

Ride On!

[sandermvanvliet]

@luciopaiva thanks for the heads up. Unfortunately this will completely break RoadCaptain as it relies on communicating directly with Zwift as if it were Zwift Companion.

Are there any plans to extend some way of communication or interoperability for 3rd party developers?

[sandermvanvliet]

Do you perhaps have a Zwift (and possibly Zwift Companion) version number from when encrypted mode is supported? I think that would allow me to inform users when they start the app so that it’s clear why RoadCaptain may not be working.

I really appreciate the heads-up, much better than things just stopping to work 👍

[luciopaiva]

Are there any plans to extend some way of communication or interoperability for 3rd party developers?

There is an ongoing discussion about this, but no updates yet, unfortunately. We'll make sure to keep you posted.

Do you perhaps have a Zwift (and possibly Zwift Companion) version number from when encrypted mode is supported?

Game client: v1.26.2 Zwift Companion: v3.37.0

[sandermvanvliet]

Thanks for the update. Is there an ETA when these versions land?

[mayfield]

July 20th

[felipenasc]

@luciopaiva thanks for the heads up. Unfortunately this will completely break RoadCaptain as it relies on communicating directly with Zwift as if it were Zwift Companion.

Are there any plans to extend some way of communication or interoperability for 3rd party developers?

Hi @sandermvanvliet , The encryption won’t break RoadCaptain during this initial phase. RoadCaptain will continue to work for some time after July 20 because the game client will fallback to legacy mode if the other peer (RoadCaptain in this case) is not yet encrypted. We have done this to provide some leniency. That will happen naturally with RoadCaptain, so there’s no need to change anything in the code right away - it will just work.

That said, we will be enforcing encryption at some point in the future. When that happens, the game will deny unencrypted connections. As mentioned in the original post, we are internally discussing if, and how, we can help community projects go through the change and continue working - and that includes RoadCaptain.

Nevertheless, we are doing the best we can because we truly believe community projects like yours are awesome. You can rest assured that suggestions from community members, like yourself, will be reviewed and considered as we look to evaluate the next phase of this encryption process.

[felipenasc]

Thanks for the update. Is there an ETA when these versions land?

@sandermvanvliet On July 20th, all Zwift platforms will be running with encryption. RoadCaptain will not lose the ability to communicate unencrypted though, as the game client will fallback to legacy mode for the time being.

[sandermvanvliet]

@felipenasc thanks for the additional clarification 👍

If you and the team at Zwift are looking for further input you can reach me (more) directly at zwift@codenizer.nl

[sandermvanvliet]

@felipenasc @luciopaiva @mayfield it looks like even with latest versions of Zwift and Zwift Companion sending turns from ZC to Zwift doesn’t work reliably. Sometimes you can’t change direction at all, sometimes only once and other times you can switch directions repeatedly until you reach the turn.

RoadCaptain isn’t able to send turns at all, command is sent but Zwift ignores it completely.

[mayfield]

... command is sent but Zwift ignores it completely.

I expect your remarks will suffer the same fate from Zwift HQ, but I'd love to be proven wrong. :smirk:

[sandermvanvliet]

One can hope…

[sandermvanvliet]

Good news, RoadCaptain now supports the encrypted connection introduced in Zwift 1.27.

A new release will be out soon!

[sandermvanvliet]

I’m keeping this issue open for a while longer but effectively this problem is solved as of release 0.6.6.0 which supports the encrypted connection.

For other 3rd party devs: I’ve documented the encryption process here

[sandermvanvliet]

@felipenasc @luciopaiva @mayfield it looks like even with latest versions of Zwift and Zwift Companion sending turns from ZC to Zwift doesn’t work reliably. Sometimes you can’t change direction at all, sometimes only once and other times you can switch directions repeatedly until you reach the turn.

RoadCaptain isn’t able to send turns at all, command is sent but Zwift ignores it completely.

@felipenasc @luciopaiva pinging again here, I was wondering if there is any progress on the commands not working issue?

[luciopaiva]

@felipenasc @luciopaiva pinging again here, I was wondering if there is any progress on the commands not working issue?

Hey @sandermvanvliet, I just learned that the game team has identified the problem and a fix is on the way, but it still may take some time until it's released. Nevertheless, I want to bring your attention to this post which explains a workaround in case you want to implement it in RC. Just be aware that if you do, RC will break again when the game team releases the fix - when that happens, you will need to revert the workaround. One possible idea is to activate the workaround only if game version 1.27.0 is detected (well, and also 1.28.0 which is already closed and almost out, I believe).

[sandermvanvliet]

Thanks for letting me know @luciopaiva. In fact I’ve already got specific handling for three-way turns (Zwift will only send two commands in these cases…) so implementing another one for these game versions should definitely be possible.

[juanplopes]

Hi @sandermvanvliet, there is a possible workaround while it is being decided how/when this fix is going live. For now, whenever you have to send LEFT, RIGHT, or STRAIGHT turn commands, instead issue a sequence of commands, like the following:

if LEFT is among the turn options:
    STRAIGHT' = LEFT, LEFT, RIGHT, STRAIGHT
else:
    STRAIGHT' = RIGHT, RIGHT, LEFT, STRAIGHT

LEFT' = LEFT, LEFT
RIGHT'= RIGHT, RIGHT

Right now, you wouldn't need to include STRAIGHT in the list, as the Game Client currently ignores it, but by doing so, you prepare your code to work correctly when a fixed version is released.

[sandermvanvliet]

I’ll test this to see if that works out.

[sandermvanvliet]

See #98, it works 👍

[mayfield]

Hi @felipenasc and @luciopaiva,

Perhaps you might be able to help me understand the duplicate server connection handling. In light of the encryption changes I built a game client that can connect and receive PlayerState packets. My goal is to just have this connection act passively. Essentially to send player state packets that are in the form of...

player_id: <id of logged in user>
player_state:
    watching_player_id: <id of logged in user / or user they are watching in their real game client>
    just_watching: true,

And thus indicate to the servers that they should send me nearby player states for the watching_player_id. However, I've not been able to find a suitable recipe that allows me to get the data stream my application needs and avoids triggering the servers to kill one of the two game connections established for this player_id.

Using a different login/player_id works fine of course, but I'd really like to avoid making my users create a dummy account if there is a cleaner way.