scubajorgen / TomTomWatch

Tool for exporting GPS files from TomTom Sports Watches
84 stars 18 forks source link

Custom workout #17

Closed sbouth closed 3 years ago

sbouth commented 4 years ago

Hey,

is there any plan to add support to custom workout ? it seems we need to reverse engineer 0x00BEnnnn files according to your documentation. Do you have some information about it ?

(this is a feature I would llike to add also on my android app and i'am not sure how difficult it is to reserve engineer it)

Thx

scubajorgen commented 4 years ago

Reverse engineering isn't that complex. The format of this file is protobuf. Basically it is a very efficient way (in terms of size) to encode/decode data/datastructures, developed by Google. There are plenty of tools by which you can read and decode protobuf files (e.g. https://protogen.marcgravell.com/decode). From there you have to compare the data with what you see in TomTom to make sense of what the fields (identified by a number) represent. Knowing the structure, you can make the .proto protobuf definition file, like in my project files like /various/protobuf/route.proto. Then you can use the Google protobuf tooling (compiler) to generate code for encoding/decoding protobuf data in virtually any programming language you like. I put one of those 0x00be files through the decoder. I can have a try to make the .proto.

scubajorgen commented 4 years ago

I reverse engineered the workout file structure. I am working at least on being able to display the workouts in the watch, in a separate branch '/feature/workout'. I am thinking of a way to create own workouts. The protobuf .proto file is in /various/protobuf/workout.proto. The only thing I did not find out yet is how to discern between cycle and running workouts. It must be somewhere in the data...

sbouth commented 4 years ago

So GREAT !!! thanks :)

I'm going to have a look and try it with my own workout.

sbouth commented 4 years ago

I have tested createWorkoutListFromData with success :) That's nice to be able to get an overview without reading all files (not sure if the same is possible for routes).

From my workout list I found that:

Those values look strange so I'm not sure if it's relevant.

scubajorgen commented 4 years ago

Thanx. I found out the same yesterday evening :-) and incorporated this stuff in my code this very morning, including some nifty sorting of the list. Indeed, weird looking values... And int64... But it seems to work. Also for custom workouts If you've got a clue on unknown9 in the same structure, please let me know. Mystery to me... I intend to add some functionality to upload and download workouts from/to JSON, YAML or XML, though a neat UI would be nicer (however, that's quite a change....)

sbouth commented 4 years ago

I can see that your workout branch is going well from last commits :) I wonder if 0x00BE0000 file must be maintained by the application (desktop or mobile) or if it's managed automatically by the watch.

I mean, when Adding/Removing or modifying a 0x00BEnnnn file, does the watch update itself 0x00BE0000 automatically or is it necessary to rewrite "manually" the 0x00BE0000 ?

Do you know is some of 0x00BEnnnn are reserved or read only ? It seems that the official mysports TT app enables only 5 customs workout for cycling + 5 customs workout for running and keep all other workout unchanged.

EDIT: For unknown9, it seems to be the workout file size

scubajorgen commented 4 years ago

0x00BE0000 isn't maintained by the watch. First, it contains data that cannot be derived from the workouts, like the RUNNING/CYCLING activity. Second, given the structure I think it makes more sense that is not generated by the watch: it contains an overview (needed for the menu without the need for reading all the 0x00BEnnnn files) with references to the workout files. To be sure I checked it: wrote an addtional 0x00BE0033 file to the watch 'by hand', disconnected the watch from the PC, reconnected, downloaded 0x00BE0000: it doesn't contain a reference to the new file.

I believe there is no reservation. I changed 0x00BE0001 using tomtom mysports (I altered the intensity level). After that it became 0x00BE0036 or so, resulting in a 'missing' 0x00BE0001. It might be that there is a limit to the number of files, e.g. max 5 to each type. I guess I found something similar: a limit of 15 routes (not entirely sure...). I guess there is only one way to find out :-)

Thank you so much for the file size hint. It gave me a headache yesterday. I assumed it to be bitfields, but I could not find it to be dependent consistently on anything... So I abandoned the assumption. This is the last field I was looking for :-). The other unknowns seem to be constant...

There is some duplication of data: name and description of the workout is in 0x00BE0000 as well as in the workout file. I am working on a template for the list including the workouts int which the data is normalized (it can be deserialized from a JSON file). Based on the template I will be able to generate the watch files. Maybe you can reuse...

sbouth commented 4 years ago

Your welcome for the file size, I'm glad I could help at least for this.

Thanks for your reply about 0x00BE0000, it's clear.

scubajorgen commented 4 years ago

Hum.. The 16 byte arrays appear to be MD5 hashes. The one that only occurs in 0x00BE0000 and not in the workout files is the MD5 hash of the workout file. If is not correct, the workout won't show up on the watch. The MD5 hash that occurs in the 0x00BE0000 and the workout files is not clear to me. It does not appear to have effect.... I am working on an update of the code....

sbouth commented 4 years ago

I had no time to work on this project recently but something came to my mind. What's happen to the other id when changing only the intensity of a workout (from the official site and app) ? I tought it could stay unchanged but really I have not tried and don't know....

scubajorgen commented 4 years ago

Good thinking. My first idea was also that the other 'ID' was an MD5 on part of the content, for example on the workout steps. However, if you change them, the 'ID' remains unaltered. Its value does not seem to have any effect on the validation by the watch, even if you make the value in 0x00BE0000 different from the value in the corresponding 0x00BEnnnn workout file. I therefore left it as 'UID' identifying the workout, but its meaning is unclear.

The MD5 hash on the workout file must be correctly stated in 0x00BE0000, otherwise the workout won't show on the watch, as I discovered. My upload-from-JSON function sometimes rearranges the list of texts leading to a valid workout but a different MD5.

Another finding: if you upload more than 5 workouts of the same type to the watch, only the first 5 are shown. Only 5 items fit on the display. The watch doesn't appear to have a scroll function. That limits the workouts to 6x5=30 workouts per activity (running/cycling). I will add a check on max 5 workouts in the workout validation.

My software now supports download to JSON and upload from JSON, including redefinition of the HR Zones, as a 'low budget' alternative to a fancy user interface.

scubajorgen commented 3 years ago

Functionality to export and import custom workouts released in version 1.6