Open victorypoint opened 1 year ago
A few more notes. Using https://www.fitfileviewer.com/, the Session and Event tables of QZ FIT file show a start timestamp of 2:14:03pm. That could possibly indicate there is missing data from 2:14:03 - 2:17:01pm in the QZ file.
seems a bug in the garmin fit library. If you look in the fitfileviewer website, the session record has a good timestamp, that's because, instead of using the date object from the fit library, i'm calculating it myself. also the lap timestamp is wrong, and it's using the fit object too...so i have to use my method only.
Question: if I create a windows version with the patch, can you help me doing a test and verify that it's all good?
@victorypoint when it's ready you will see a new windows version here https://github.com/cagnulein/qdomyos-zwift/actions/runs/4267692357 (bottom of the page) it will be ready in 30 minutes
@cagnulein, thank-you! Very excited to test this out. That's interesting about the timestamp bug in the Garmin FIT SDK. I'm curious if you're using the latest 21.105 SDK, Feb. 2, 2023? - https://developer.garmin.com/fit/download/
no I updated the last time some months ago. I usually don't use never the last version due to possible new bugs. When I find something stable I usually stick with it ;)
Roberto Viola Software engineer and open source enthusiast http://robertoviola.cloud
Il giorno sab 25 feb 2023 alle ore 02:56 Al Udell @.***> ha scritto:
@cagnulein https://github.com/cagnulein, thank-you! Very excited to test this out. That's interesting about the timestamp bug in the Garmin FIT SDK. I'm curious if you're using the latest 21.105 SDK, Feb. 2, 2023? - https://developer.garmin.com/fit/download/
β Reply to this email directly, view it on GitHub https://github.com/cagnulein/qdomyos-zwift/issues/1311#issuecomment-1444905535, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAALYWDNKGNPEJ7BHASC3TLWZFRDFANCNFSM6AAAAAAVHIEN2Y . You are receiving this because you were mentioned.Message ID: @.***>
@victorypoint ready https://github.com/cagnulein/qdomyos-zwift/suites/11196536723/artifacts/572412013
Thank-you. I started doing some testing. Go to bed! π΄π
@cagnulein, using new Windows QZ build v2.12.73, I repeated my test as before using QZ fake treadmill mode::
Here are detailed timestamps per FIT table:
Zwift timestamps:
File_id - time created - 10:29:21pm, 1 record
Event - start timestamp - 10:33:05pm, stop timestamp - 10:40:17pm, 2 records
Lap - timestamp - 10:40:16pm, start_time - 10:33:05pm, 1 record
Session - timestamp - 10:40:17pm, start_time - 10:33:05pm, 1 record
Activity - timestamp - 10:40:17pm, 1 record
Record - first timestamp - 10:33:06pm, 432 records - this appears to be when QZ speed preset button was pressed
QZ timestamps:
File_id - time created - 10:33:04pm, 1 record
Event - timestamp - 10:33:04pm, 1 record, no stop timestamp
Lap - timestamp - 10:33:04pm, start_time - 10:33:04pm, 1 record
Session - timestamp - 10:33:04pm, start_time - 10:33:04pm, 1 record
Activity - timestamp - 10:33:04pm, 1 record
Record - first timestamp - 10:35:58pm, 419 records
FIT files attached. test1.zip
sorry @victorypoint I found now the original issue, it's not fault of the fit library, it's just my fault :( New build here in the bottom https://github.com/cagnulein/qdomyos-zwift/actions/runs/4268708107
@victorypoint in case you missed, new artifact is ready
@cagnulein, this update is great! I retested as before. Here's the results:
Looking at the FIT tables using https://www.fitfileviewer.com/, I noticed both FIT files have consistent data structure, however the QZ Event table has no stop timestamp record whereas the Zwift table does. Is this important?
I will continue to do more testing.
ok let me know when i can merge this!
Il giorno sab 25 feb 2023 alle 23:41 Al Udell @.***> ha scritto:
@cagnulein https://github.com/cagnulein, this update is great! I retested as before. Here's the results:
- Launch QZ in paused state - 2:38pm
- Launch Zwift and begin a manual run - 2:42pm - avatar stands still waiting for speed
- Hit QZ Start button - 2:43pm - avatar stands still waiting for speed
- Hit QZ preset speed button 10kph - 2:45pm - avatar begins to move
- Hit QZ Stop button, end Zwift run, exit both QZ and Zwift - 2:55pm
- Examine Zwift and QZ FIT files:
- Zwift starting timestamp - 2:45:07pm - this appears to be very close when QZ speed preset button was pressed
- QZ starting timestamp - 2:45:04pm - this also appears to be very close QZ speed preset button was pressed
Here's the original Zwift FIT file elevation graph overlayed with the elevation corrected FIT file:
[image: image] https://user-images.githubusercontent.com/63697253/221381711-65686043-521e-492f-aecb-06cb22bb6fa4.png
I will continue to do more testing.
β Reply to this email directly, view it on GitHub https://github.com/cagnulein/qdomyos-zwift/issues/1311#issuecomment-1445220338, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAALYWAR7LSB4PW6S2BOBYLWZKDBNANCNFSM6AAAAAAVHIEN2Y . You are receiving this because you were mentioned.Message ID: @.***>
-- Roberto Viola Software engineer and open source enthusiast http://robertoviola.cloud
@cagnulein, my next test discovered a problem with how QZ calculates elevation gain. I'll post the issue here but let me know if you want a new ticket for this.
For this test, I wanted to compare how QZ calculated elevation compares with the terrain elevation for a given course. To get a terrain elevation graph, I used bike mode and mapped the entire 23K Volcano Climb course in a FIT file. Next, I switched to run mode (using the bike pairing meetup trick) and repeated the course using auto-inclination, then merged the QZ elevation into the resulting Zwift FIT file. Here's the 2 graphs overlayed. You can see the Zwift bike elevation in blue and QZ run elevation in green. From what I can tell, the ascent elevations are very close, however QZ appears to be shallowing the decent.
Note that I used fake treadmill for this test with 'Zwift inclination gain' = 2. For comparison, here is an elevation graph of Zwift bike elevation overlayed with QZ bike calculated elevation.
So, it appears that QZ is treating decent (negative elevation) with a different calculation than ascent (positive elevation).
yes because zwift send the negative inclination to half :) you have to enable the zwift negative incliantion double setting. There are some youtube videos about this zwift issue too :)
@cagnulein, I repeated this test using the longer Mega Pretzel course just to confirm it wasn't isolated to a single course. Again I'm finding that QZ is treating decent (negative elevation) with a different calculation than ascent (positive elevation). Here's the 2 graphs overlayed - Zwift bike elevation in blue and QZ run elevation in green. Again, I'm using fake treadmill with 'Zwift inclination gain' = 2.
Also, here is the elevation graph of Zwift bike elevation overlayed with QZ bike calculated elevation.
check my above comment @victorypoint
yes because zwift send the negative inclination to half :) you have to enable the zwift negative incliantion double setting. There are some youtube videos about this zwift issue too :)
Thanks! .. desperately poking around in QZ looking for this setting now...
yes because zwift send the negative inclination to half :) you have to enable the zwift negative incliantion double setting. There are some youtube videos about this zwift issue too :)
Sorry, I can't find this setting. Why exactly is Zwift sending negative inclination to half and positive inclination to full?
bike options, double negative inclination. zwift does this because, IMHO, the bikes with negative inclination will be uncontrollable (because you will have super high cadence during downhill)
@cagnulein, found it under Bike options - 'Double Negative Inclination'. So it's a Zwift bug?
yes confirmed :)
yes confirmed :)
Ok thanks. Will retest.
@cagnulein, so with QZ 'Double Negative Inclination' turned on this time, unfortunately I'm getting the same elevation graphs as before. I'm thinking maybe there's another setting that needs to be set in QZ. I'm using fake treadmill with 'Zwift inclination gain' = 2 as before with Wahoo Treadmill device.
Logs and FIT files attached. bike vs run mode.zip
oh that's strange, let me check this!
Il giorno mar 28 feb 2023 alle 03:40 Al Udell @.***> ha scritto:
@cagnulein https://github.com/cagnulein, so with Zwift 'Double Negative Inclination' turned on this time, unfortunately I'm getting the same elevation graphs as before. I'm thinking maybe there's another setting that needs to be set in QZ. I'm using fake treadmill with 'Zwift inclination gain' = 2 as before with Wahoo Treadmill device.
[image: GPXSee 2023-02-27 10_33_45 PM] https://user-images.githubusercontent.com/63697253/221738963-b2edf4df-9b6d-4a64-9672-3ae1d652bac4.png
Logs and FIT files attached. bike vs run mode.zip https://github.com/cagnulein/qdomyos-zwift/files/10846032/bike.vs.run.mode.zip
β Reply to this email directly, view it on GitHub https://github.com/cagnulein/qdomyos-zwift/issues/1311#issuecomment-1447481782, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAALYWGMA5EGH7774UW7WV3WZVQSZANCNFSM6AAAAAAVHIEN2Y . You are receiving this because you were mentioned.Message ID: @.***>
-- Roberto Viola Software engineer and open source enthusiast http://robertoviola.cloud
@victorypoint i'm an idiot, i'm applying the zwift negative inclination only to the bikes. That's why you're seeing this. Ok so now I'm doing 2 modifications
done @victorypoint you can check this new modification here https://github.com/cagnulein/qdomyos-zwift/actions/runs/4290568258 (always on the bottom for the artifact)
done @victorypoint you can check this new modification here https://github.com/cagnulein/qdomyos-zwift/actions/runs/4290568258 (always on the bottom for the artifact)
Ok awesome. I will test this morning. One thing I noticed yesterday which I should have posted is the erg grade was different than the transmitted incline. Not sure if they are supposed to match? For example:
calculated erg grade 3.99" changeInclination 5.32 true 1 0 calculated erg grade 4.08" changeInclination 5.44 true 1 0 calculated erg grade 4.095" changeInclination 5.46 true 1 0 calculated erg grade 4.2" changeInclination 5.6 true 1 0
no they shouldn't there is a radian degrees conversion in the middle
@cagnulein, I just completed a bike test using the new QZ build, same course and settings. Here's the elevation graph showing both Zwift elevation in blue and QZ calculated elevation in green. For bike these graphs should match close?
yes do you have the debug log? I guess I'm missing something
hah wait, for the bike the issue is the speed, the speed can't match to zwift, did you check? because zwift calculates the speed based on power @victorypoint
hah wait, for the bike the issue is the speed, the speed can't match to zwift, did you check? because zwift calculates the speed based on power @victorypoint
Ok, that sounds like the problem then. I'm just setting speed to 10kph in QZ which translates to around 35-38kph for bike
yes I guess the only test that matters it's with the treadmill mode. Sorry if I didn't raise this first, but I just thought about this right now!
for the bike the issue is the speed, the speed can't match to zwift, did you check? because zwift calculates the speed based on power
No worries. I'll test this later after my ZLDR runs.
@cagnulein, this is looking better. Here's Zwift vs QZ run elevation on same course..
And Zwift bike elevation vs QZ run elevation, same course. The graph profiles look almost identical but I don't understand why they're offset about 20 metres?
green is always qz in the second chart? if so i don't understand either the offset. do you have the debug log?
@cagnulein, I think I understand... I started at elevation
green is always qz in the second chart? if so i don't understand either the offset. do you have the debug log?
Yes, QZ is green. Hang on... I think I know why.. retesting...
ok meanwhile i'm going to bed :D
Il giorno mar 28 feb 2023 alle 21:37 Al Udell @.***> ha scritto:
@cagnulein https://github.com/cagnulein, I think I understand... I started at elevation
green is always qz in the second chart? if so i don't understand either the offset. do you have the debug log?
Yes, QZ is green. Hang on... I think I know why.. retesting...
β Reply to this email directly, view it on GitHub https://github.com/cagnulein/qdomyos-zwift/issues/1311#issuecomment-1448881445, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAALYWBXLQLZV3XSP4PUL2LWZZOZFANCNFSM6AAAAAAVHIEN2Y . You are receiving this because you were mentioned.Message ID: @.***>
-- Roberto Viola Software engineer and open source enthusiast http://robertoviola.cloud
@cagnulein, I retested... so it looks like the vertical offset between the 2 elevation profiles is due to Zwift runner having different elevation at the beginning of the run (FIT file). So QZ calculates the entire course elevation relative to the starting elevation. In this case, the difference is:
But also note when comparing the 2 graphs that the horizontal start is not exactly the same. I think this is due to the difference in speed between bike and run tests. But this also adds a bit more elevation difference between the 2 profiles.
So it seems that when running with auto-inclination, the QZ elevation profile will always be relative to the QZ starting elevation. Ideally, this starting elevation should be the terrain elevation for that location but we have no way of getting this information (Zwift doesn't send it, just % grade).
Is there's no way to reset the QZ start elevation unless QZ is stopped and started (setting everything back to 0)? So somehow the user would need to stop/start QZ at the beginning of each run in order to set elevation to 0. But this sets start elevation to 0 which is likely not always the actual terrain elevation.
Is there's no way to reset the QZ start elevation unless QZ is stopped and started (setting everything back to 0)? So somehow the user would need to stop/start QZ at the beginning of each run in order to set elevation to 0. But this sets start elevation to 0 which is likely not always the actual terrain elevation.
Yes there is no way to sync them precisely, you should reset elevation to qz (pressing stop) and then start from a Zwift elevation of 0 (very unlikely)
@cagnulein, I'm just throwing out some ideas on this FIT file sync issue. Let me know your thoughts on how we could simplify or automate the process.
We know that syncing Zwift and QZ FIT files and merging them so that QZ elevation replaces Zwift elevation solves the problem of Zwift elevation for running.
So to get synced FIT files, the QZ Start button must be triggered at the beginning of each Zwift activity, and the QZ Stop button must be triggered at the end of the Zwift activity.
How can we trigger QZ actions (Stop, Start) automatically to coincide with Zwift activities? In my case, QZ is always running in the background of full-screen Zwift, so accessing the buttons to stop/start QZ is not easy.
Once the 2 FIT files are complete, then merging them currently requires a command-line process. The problem is Zwift and QZ FIT filenames and timestamps don't match, making it difficult to match up the files for merging. For example:
A Zwift FIT filename has the form "2023-02-24-11-36-34.fit". It's created when the Zwift activity is ended and is named using the file creation data/timestamp.
A QZ FIT filename has the form "Fri Feb 24 11_33_51 2023.fit". It's created when QZ is stopped and uses the file creation/timestamp.
Here's an example of QZ files recorded for a Zwift activity. Filenames and timestamps appear to be generated at various times as the code executes:
- How can we trigger QZ actions (Stop, Start) automatically to coincide with Zwift activities? In my case, QZ is always running in the background of full-screen Zwift, so accessing the buttons to stop/start QZ is not easy.
maybe with intent you can automate it with tasker or similar? or maybe we can put a setting that when the speed is greater than 0 it start a new activity and it stop when the speed is equal to 0?
- Once the 2 FIT files are complete, then merging them currently requires a command-line process. The problem is Zwift and QZ FIT filenames and timestamps don't match, making it difficult to match up the files for merging. For example:
since it's a terminal application the both files could be passed as an arguments, don't?
maybe with intent you can automate it with tasker or similar? or maybe we can put a setting that when the speed is greater than 0 it start a new activity and it stop when the speed is equal to 0?
That's a great idea but I can foresee problems as users would be doing a bit of running before an event starts for example (avatar running on virtual treadmill). I like the idea of a tasker or AHK hotkey or bluetooth button press. Will test it out.
since it's a terminal application the both files could be passed as an arguments, don't?
I have an idea to test...
@cagnulein, I adapted these Java utilities to merge Zwift FIT file with QZ FIT file to add Zwift location data to QZ. They both use the Garmin FIT SDK.
Fork of nickrodie/Zmerge repo - https://github.com/victorypoint/Zmerge
Fork of nickrodie/garmin-zwift-merge - https://github.com/victorypoint/qz-zwift-merge
Is it worth incorporating this code into QZ or just run independently?
Wow great work! I feel difficult to incorporate this in QZ directly, It will be better to keep it as a separate repository as I'm doing also for the Companions. Do you agree?
Wow great work! I feel difficult to incorporate this in QZ directly, It will be better to keep it as a separate repository as I'm doing also for the Companions. Do you agree?
Sure. No worries. It will be easier now at least to set up a batch merge process.
... or maybe we can put a setting that when the speed is greater than 0 it start a new activity and it stop when the speed is equal to 0?
@cagnulein, I think this may be worth testing. I'm liking the idea more and more.
the only thing is that how handle the pause event :) btw can i merge the main modification of this ticket ? i guess it's working and also it's safe
Il giorno gio 2 mar 2023 alle 20:00 Al Udell @.***> ha scritto:
... or maybe we can put a setting that when the speed is greater than 0 it start a new activity and it stop when the speed is equal to 0?
@cagnulein https://github.com/cagnulein, I think this may be worth testing. I'm liking the idea more and more.
β Reply to this email directly, view it on GitHub https://github.com/cagnulein/qdomyos-zwift/issues/1311#issuecomment-1452404378, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAALYWAXXJBHP6FIFJUVU4DW2DU4ZANCNFSM6AAAAAAVHIEN2Y . You are receiving this because you were mentioned.Message ID: @.***>
-- Roberto Viola Software engineer and open source enthusiast http://robertoviola.cloud
the only thing is that how handle the pause event :)
Is this what you mean? We will likely need Settings-Treadmill Options-"Pause when App Starts" turned on. First Zwift activity, we press Start button start then Stop button when done. Then reset QZ for next activity by pressing Back and then Pause. Or we leave "Pause when App Starts" turned off. Then we press Pause then Start at beginning of activity.
btw can i merge the main modification of this ticket ? i guess it's working and also it's safe
Yes, it's working well!
Yes, it's working well!
merged!
We will likely need Settings-Treadmill Options-"Pause when App Starts" turned on. First Zwift activity, we press Start button start then Stop button when done. Then reset QZ for next activity by pressing Back and then Pause. Or we leave "Pause when App Starts" turned off. Then we press Pause then Start at beginning of activity.
yeah I mean I was looking to find a way to achieve this without adding any new settings. If I'm right, QZ elapsed timer is not counting if the speed is == 0, if this is true the start trigger is ok for the fit file. For the stop instead it's tricky because there is no automatic way to stop it. But I was thinking: is it really necesseary? I mean since the Zwift activity has an end, we can simple discard all the remaining qz activity from the zwift one (the qz activity will be probably longer than the zwift one, but we can simply cut the remaining off, isn't it?)
@cagnulein, now that we have a consistent way to synchronize that start of activities between Zwift and QZ (ensure speed is 0 then increase to start FIT file recording at the same time), I'm finding a strange inconsistency with the start timestamp in the QZ FIT file. This is affecting the ability to accurately merge data between the two FIT files (e.g. latitude, longitude, elevation, power, etc).
For example, here's results of a recent test using QZ fake treadmill mode:
I've retested this several times and always arrive at a strange delayed QZ starting timestamp. Any help resolving this is greatly appreciated.
FIT files attached. test4.zip