Refactor protobufs

[ursoft]

Just decompiled all used protobufs from the 100317+ binary code and put results into my fork https://github.com/ursoft/zwift-offline Original names (from debug info and strings) for messages, duplicates and other things are commented inside corresponding proto-files. Some data types may be decompiled with errors (intXX<->sintXX<->uintXX, string<->bytes etc), so all changes need to be checked on real data.

[oldnapalm]

I've been using the refactored protobufs, no issues so far. Also have various success reports of people using the zca branch.

Is there any specific test you recommend or just normal use?

Thanks for the awesome work.

[ursoft]

Yesterday I've improved some protobufs in my fork. There are many questions, most of them you can find inside remarks (investigation is still in progress). For example, PlayerState.route(39) is definitely 32 bit, not 64 bit (this clarification causes some ugly changes in python, neg 64 Attribute.number_value -> neg 32 PlayerState.route).

The only extended test I could recommend - is ride with a friend. As I can see, sometimes (only when ZO used), my friend suddenly "jumps" forward or backward on minute or two. I'm going to investigate this issue soon - maybe you will find the bug before me. This is the main reason why I'm trying to decompile protobufs.

I'm sorry for my poor English, hope you're still understanding me :-)

[oldnapalm]

When you say "jumps forward or backward one minute or two" do you mean the avatar or only the name in nearby list?

I usually see the names switching positions in nearby list when riding with ghosts, mainly when the road "changes direction" (e.g. you are going north, then the road has a 180º curve and you start going south).

Some time ago an user reported a problem of the remote player avatar disappearing and appearing in a different location, but apparently it was fixed with this change https://github.com/zoffline/zwift-offline/commit/c9483933751c70b4443766eed95e2c2c4d880b97 (or the user gave up testing).

I only have one trainer so it's difficult to test riding with a friend.

[ursoft]

mean the avatar or only the name in nearby list

Both: my friend's avatar is disappeared from my view and the time in the nearby list changes from seconds to minutes. We rode together, all the turns were the same, the speeds were almost equal and after "jump" we both see the same environment (trees, buildings etc) except for peer avatar and broken nearby list.

[oldnapalm]

Just a wild guess, I noticed you changed PlayerState.heading from int64 to uint32, are you sure about that? I ask because I've seen heading as negative value for angles in 3rd and 4th quadrants (although it wasn't in Zwift).

[ursoft]

I only have one trainer so it's difficult to test riding with a friend.

You need Android app named "nRF Connect". Create GATT Server for Cycling Power in nRF Connect or use my cfg:

<server-configuration name="Power80W">
   <service uuid="00001818-0000-1000-8000-00805f9b34fb">
      <characteristic uuid="00002a63-0000-1000-8000-00805f9b34fb" value="00005000">
         <descriptor configure="CCCD"/>
         <permission name="READ"/>
         <property name="NOTIFY"/>
      </characteristic>
   </service>
</server-configuration>

also you need a macro for nRF Connect:

<macro name="Send Power notifications every second" icon="UPLOAD">
   <sleep description="This sample will show how to automate sending every second a notification from the Android device." timeout="300" />
   <sleep description="Make sure the connected device enabled CCCD for 00002a63-0000-1000-8000-00805f9b34fb characteristic." timeout="200" />
   <send-notification description="Send 0x00005000 as notification from 00002a63-0000-1000-8000-00805f9b34fb." characteristic-uuid="00002a63-0000-1000-8000-00805f9b34fb" service-uuid="00001818-0000-1000-8000-00805f9b34fb" value="00005000"/>
   <sleep description="Now sleep some time, so the notification is sent every second." timeout="400"/>
   <sleep description="To run the macro in a loop check the &#8644; option above, and run macro with &#9654; button." timeout="100"/>
</macro>

upd: nRF advertiser also need to be set up to publish your phone as 0x1818 Power Source

[ursoft]

noticed you changed PlayerState.heading from int64 to uint32

Thanks, it's my bug, need to be fixed. In binary code:

      case 13:
          ...
          done = (google::protobuf::internal *)google::protobuf::internal::VarintParseSlow64(
                                                 (google::protobuf::internal *)TagFallback,
                                                 (const char *)a2,
                                                 (unsigned int)a3);
          this->m_heading = (int)a2; // 64 -> 32 bit

this code was autogenerated from proto and I do not know why this->m_heading is truncated after read...

[oldnapalm]

Merged your latest changes. Just missed one change here after route changed to 32 bit (otherwise Watopia pace partners will show with null route) https://github.com/ursoft/zwift-offline/commit/8356bef1b6fc059b626ecb5406e5ea3323023c4c#diff-f4053a7fcb3c73ca8b58c6fb646b8f0e2c467edfad4759aa3a241f09b3fe2ca2L315

Edit: I mean, after Attribute.id changed to 64 bit

[ursoft]

Let's call that bug "Peer disappear" (tried to reproduce it today with no luck), I'll try to describe it here: https://github.com/ursoft/zwift-offline/issues/1

Another "interesting" issue about protobufs or profile I call "Circus" https://github.com/ursoft/zwift-offline/issues/2

[ursoft]

I mean, after Attribute.id changed to 64 bit

As I can see in zwift::protobuf::ProfileAttribute::_InternalParse, ProfileAttribute.Id is 32 bit But in previously named 'PacerSetting' it was 64 bit too...

[oldnapalm]

As I can see in zwift::protobuf::ProfileAttribute::_InternalParse, ProfileAttribute.Id is 32 bit But in previously named 'PacerSetting' it was 64 bit too...

Yes, but with 'PacerSetting' we checked if id == 3273955058 for Watopia pace partners and when you changed to 32 bit you also changed to if id == -1021012238

Another "interesting" issue about protobufs or profile I call "Circus" ursoft#2

If it's not related to steering, maybe it's about the packets synchronization. If you receive a PlayerState packet with given speed and roadTime values for a remote player, and the next packet has a roadTime value which is not consistent with the "predicted" position based on the previous packet, the avatar will make these maneuvers to try to synchronize the position. That's what I tried to fix with the change of updating the online players states only once a second, but maybe it wasn't a good approach, maybe online players should be updated more often.

[ursoft]

if id == 3273955058

BTW, do you know the string which crc32=3273955058 ?

maybe online players should be updated more often

Thanks, I'll try to test different freqs

[oldnapalm]

I don't know, got that value raw decoding the first pace partners profiles (ids 24), the current ones (ids 45) have the route in 1766985504. I had no idea it was a string crc32.