search

cagnulein / QZCompanionNordictrackTreadmill

Companion App of QZ for Nordictrack Treadmills and Bikes
https://www.qzfitness.com/
GNU Affero General Public License v3.0
50 stars 14 forks source link

iFit Beta Equipment Software Breaks QZCompanion #103

Closed keatontaylor closed 3 weeks ago

keatontaylor commented 3 months ago

This is just a PSA the beta version of the latest software for the nordictrack hardware appears to no longer write to the wolflogs directory. Thus completely breaks fetching the stats from the bike via that file.

cagnulein commented 3 months ago

hi @keatontaylor yes i'm aware of this. I guess I should go with some OCR from the ifit app. What do you think @victorypoint ?

cagnulein commented 3 months ago

@keatontaylor are you able to leave the beta right?

keatontaylor commented 3 months ago

@cagnulein I was able to unenroll in the beta program which immediately reboots the equipment and loads the older version of the iFit software. Everything is fully functional again.

I did spend some time looking for a new file location for the bike status logs. Unfortunately, it appears that the .wolflogs for each day are still created, but it is just an empty file. So I don't think that the new software has debug/trace logging turned off.

cagnulein commented 3 months ago

Ok this weekend I will play with OCR thing if you are available to test it. What do you think?

Roberto Viola Software engineer and open source enthusiast http://robertoviola.cloud

Il giorno ven 2 ago 2024 alle ore 16:06 Keaton Taylor < @.***> ha scritto:

@cagnulein https://github.com/cagnulein I was able to unenroll in the beta program which immediately reboots the equipment and loads the older version of the iFit software. Everything is fully functional again.

I did spend some time looking for a new file location for the bike status logs. Unfortunately, it appears that the .wolflogs for each day are still created, but it is just an empty file. So I don't think that the new software has debug/trace logging turned off.

— Reply to this email directly, view it on GitHub https://github.com/cagnulein/QZCompanionNordictrackTreadmill/issues/103#issuecomment-2265475584, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAALYWAV76R6DBUSQJG6WWLZPOG4PAVCNFSM6AAAAABL3L5J6GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENRVGQ3TKNJYGQ . You are receiving this because you were mentioned.Message ID: @.*** com>

keatontaylor commented 3 months ago

Unfortunately I don't see a way to quickly go back to the upgraded app version(s) after un-enrolling.

cagnulein commented 3 months ago

ok anyway i will put in on the list since i guess more users will use the beta in the future

keatontaylor commented 3 months ago

Has there been any work in trying to fetch the data from the USB interface similar to how the iFit apps do it? I'd be curious if that might be a more reliable approach if it is something that could fairly easily be reverse engineered.

cagnulein commented 3 months ago

In order to do so I need someone with the treadmill to make some experiments :) I don't have an ifit device :) Anyway ocr will work just fine I guess

Roberto Viola Software engineer and open source enthusiast http://robertoviola.cloud

Il giorno ven 2 ago 2024 alle 19:58 Keaton Taylor @.***> ha scritto:

Has there been any work in trying to fetch the data from the USB interface similar to how the iFit apps do it? I'd be curious if that might be a more reliable approach if it is something that could fairly easily be reverse engineered.

— Reply to this email directly, view it on GitHub https://github.com/cagnulein/QZCompanionNordictrackTreadmill/issues/103#issuecomment-2265882440, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAALYWBLTJIM7EKWAKNUCSTZPPCCXAVCNFSM6AAAAABL3L5J6GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENRVHA4DENBUGA . You are receiving this because you were mentioned.Message ID: @.*** com>

cagnulein commented 3 months ago

app-debug.apk.zip @victorypoint apk for OCR, it's just stream in the logcat what it sees. If we can see all the field in the manual workout windows we are good to go!

victorypoint commented 3 months ago

@cagnulein, a good start. Looks like we got some metrics. In the logcat I see:

    Line  222: 08-07 10:36:44.131  3970  4076 I SF      : Key button change - Button happened - MPH_1
    Line  411: 08-07 10:36:53.806  3970  4076 I SF      : Key button change - Button happened - MPH_2
    Line  532: 08-07 10:37:00.582  3970  4078 I SF      : Key button change - Button happened - MPH_3
    Line  615: 08-07 10:37:06.529  3970  4072 I SF      : Key button change - Button happened - MPH_4
    Line  790: 08-07 10:37:18.094  3970  4076 I SF      : Key button change - Button happened - MPH_6
    Line  902: 08-07 10:37:23.947  3970  4072 I SF      : Key button change - Button happened - MPH_12
    Line  971: 08-07 10:37:28.058  3970  4043 I SF      : Key button change - Button happened - MPH_1
    Line 1181: 08-07 10:37:41.722  3970  4076 I SF      : Key button change - Button happened - INCLINE_2
    Line 1253: 08-07 10:37:46.690  3970  4072 I SF      : Key button change - Button happened - INCLINE_3
    Line 1318: 08-07 10:37:50.907  3970  4075 I SF      : Key button change - Button happened - INCLINE_4
    Line 1383: 08-07 10:37:54.234  3970  4076 I SF      : Key button change - Button happened - INCLINE_12
    Line 1682: 08-07 10:38:12.292  3970  4075 I SF      : Key button change - Button happened - INCLINE_NEG_3
    Line 2039: 08-07 10:38:36.015  3970  4075 I SF      : Key button change - Button happened - INCLINE_NEG_2
    Line 2100: 08-07 10:38:39.189  3970  4045 I SF      : Key button change - Button happened - INCLINE_NEG_1
    Line 2157: 08-07 10:38:42.326  3970  4075 I SF      : Key button change - Button happened - INCLINE_0
    Line 2238: 08-07 10:38:47.650  3970  4043 I SF      : Key button change - Button happened - STOP

I have TM set to metric. Made speed changes - 1.6, 2.4, 3.2, 4.8, 6.4, 12.9, 19.3. Incline changes - 2, 3, 4, 12, -3, -2, -1, 0. Then stop.

stream-logcat.zip

victorypoint commented 3 months ago

@cagnulein, full logcat when Companion screen stays open (however no OCR recording prompt appears). logcat.zip

cagnulein commented 3 months ago

app-debug.apk.zip new one @victorypoint without the alarm to avoid the crash

victorypoint commented 3 months ago

@cagnulein, new logcats from new apk testing... adb scripts.zip

victorypoint commented 3 months ago

@cagnulein, after some fiddling I got the companion prompt to appear and start recording. Logcats show more OCR messages now.. adb scripts.zip

cagnulein commented 3 months ago

what a relief! so there is hope!

Roberto Viola Software engineer and open source enthusiast http://robertoviola.cloud

Il giorno mer 7 ago 2024 alle ore 17:28 Al Udell @.***> ha scritto:

@cagnulein https://github.com/cagnulein, after some fiddling I got the companion prompt to appear and start recording. Logcats show more OCR messages now.. adb scripts.zip https://github.com/user-attachments/files/16530780/adb.scripts.zip

— Reply to this email directly, view it on GitHub https://github.com/cagnulein/QZCompanionNordictrackTreadmill/issues/103#issuecomment-2273742910, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAALYWAGBZDH46YS33ZAMOLZQI4IPAVCNFSM6AAAAABL3L5J6GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZTG42DEOJRGA . You are receiving this because you were mentioned.Message ID: @.*** com>

cagnulein commented 3 months ago

@victorypoint new one. here i'm trying to send the incline and the speed to qz on the other side with the same method as before. Let's see how it goes, probably we need several version to adjust everything. Thanks

https://github.com/cagnulein/QZCompanionNordictrackTreadmill/actions/runs/10288035241/artifacts/1786555345

victorypoint commented 3 months ago

@cagnulein, here is the logs for the new apk. logcats.zip

cagnulein commented 3 months ago

@victorypoint https://github.com/cagnulein/QZCompanionNordictrackTreadmill/actions/runs/10297587613/artifacts/1789131707 I guess this would be the one!

victorypoint commented 3 months ago

@cagnulein, logs attached for new apk. logcats.zip

cagnulein commented 3 months ago

that's great! i see everything. did you try to see if the qz app works as usual on your pc or android? only for reading metrics for now

Roberto Viola Software engineer and open source enthusiast http://robertoviola.cloud

Il giorno gio 8 ago 2024 alle 18:23 Al Udell @.***> ha scritto:

@cagnulein https://github.com/cagnulein, logs attached for new apk. logcats.zip https://github.com/user-attachments/files/16550872/logcats.zip

— Reply to this email directly, view it on GitHub https://github.com/cagnulein/QZCompanionNordictrackTreadmill/issues/103#issuecomment-2276212893, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAALYWE3YROV22QNELY43ILZQOLRFAVCNFSM6AAAAABL3L5J6GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZWGIYTEOBZGM . You are receiving this because you were mentioned.Message ID: @.*** com>

victorypoint commented 3 months ago

@cagnulein, not yet. Do I keep all QZ settings the same as before?

cagnulein commented 3 months ago

Yes, please!

Il giorno gio 8 ago 2024 alle 18:59 Al Udell @.***> ha scritto:

@cagnulein https://github.com/cagnulein, not yet. Do I keep all QZ settings the same as before?

— Reply to this email directly, view it on GitHub https://github.com/cagnulein/QZCompanionNordictrackTreadmill/issues/103#issuecomment-2276272036, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAALYWDI7VAGWOYG4R4Y663ZQOPVJAVCNFSM6AAAAABL3L5J6GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZWGI3TEMBTGY . You are receiving this because you were mentioned.Message ID: @.*** com>

victorypoint commented 3 months ago

@cagnulein, no metrics in QZ unfortunately. Exact same setup as before for QZ.

cagnulein commented 3 months ago

could you please upload log on QZ app? I would like to see if you are receiving something on the UDP socket. I don't remember if you were using all the ADB On your set up Roberto Viola Software engineer and open source enthusiast http://robertoviola.cloud

Il giorno gio 8 ago 2024 alle 19:06 Al Udell @.***> ha scritto:

@cagnulein https://github.com/cagnulein, no metrics in QZ unfortunately. Exact same setup as before for QZ.

— Reply to this email directly, view it on GitHub https://github.com/cagnulein/QZCompanionNordictrackTreadmill/issues/103#issuecomment-2276284877, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAALYWCRIDOKLBBLOD2LKHDZQOQSXAVCNFSM6AAAAABL3L5J6GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZWGI4DIOBXG4 . You are receiving this because you were mentioned.Message ID: @.*** com>

victorypoint commented 3 months ago

@cagnulein, here it is debug-Thu_Aug_8_14_08_58_2024.log

cagnulein commented 3 months ago

Thanks, I'll check it out. I'm currently running outside now

Il giorno gio 8 ago 2024 alle 19:11 Al Udell @.***> ha scritto:

@cagnulein https://github.com/cagnulein, here it is debug-Thu_Aug_8_14_08_58_2024.log https://github.com/user-attachments/files/16551327/debug-Thu_Aug_8_14_08_58_2024.log

— Reply to this email directly, view it on GitHub https://github.com/cagnulein/QZCompanionNordictrackTreadmill/issues/103#issuecomment-2276292846, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAALYWA5HNXAUYGEHAR352LZQORFVAVCNFSM6AAAAABL3L5J6GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZWGI4TEOBUGY . You are receiving this because you were mentioned.Message ID: @.*** com>

cagnulein commented 3 months ago

good news I can see everything @victorypoint ! the problem/feature is that with your version, we did everything through adb, but since now the log is not available anymore we have to revert back to the UDP socket. Maybe I can put this under the adb setting that it's already available. I will build a new one for you. Finger crossed!

victorypoint commented 3 months ago

good news I can see everything @victorypoint ! the problem/feature is that with your version, we did everything through adb, but since now the log is not available anymore we have to revert back to the UDP socket. Maybe I can put this under the adb setting that it's already available. I will build a new one for you. Finger crossed!

@cagnulein, okay sounds good. I'm using the companion that is compatible with QZ Windows and AI.Server for OCR with Zwift and ADB control of TM speed and incline. I can't recall which build it is.

cagnulein commented 3 months ago

@victorypoint fixed https://github.com/cagnulein/QZCompanionNordictrackTreadmill/actions/runs/10308626311/artifacts/1792191118

I forgot to add the changed grade and changed speed tag in the udp frames, that's why qz was discarding it :) give it a go when you have time :P

victorypoint commented 3 months ago

@cagnulein, good! It works! Just a few quick testing notes:

cagnulein commented 3 months ago

Good news :) i will investigate on both issues. thanks!

Il giorno giovedì 8 agosto 2024, Al Udell @.***> ha scritto:

@cagnulein, good! It works! Just a few quick testing notes:

OCR inaccuracies - I changed to various speeds and QZ displayed incorrect values - e.g. 4.8 on TM was 4.3 in QZ time between speed change on TM to QZ display was about 8 second delay per change

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.< https://ci3.googleusercontent.com/meips/ADKq_NYfnl4ODAHPGpBONa2z5nYLJeaWS5L5zJAXq3qFOWdp-4_8D3v86k0OJxK9p695AKQrGReCNc3duwHRLFCuN6qLqn27ppIOFq8oO1mxbg-78OmocFl6Cuxo4SIkqizHuTt1LZadnKFJga0S85wtCEcC5S3seuAuz1yH8BbBXa4ZVjVh4DuX5Cxayk5RGPJbHgJCbGeEDK3ty6esnEu6WKpVjK0rET_2J0F5Y0aZ2aF3yMJhknl3kws=s0-d-e1-ft#https://github.com/notifications/beacon/AAALYWDNYWDNESNE5TVKRGTZQPKORA5CNFSM6AAAAABL3L5J6GWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTUHWKGHG.gif>Message ID: <cagnulein/QZCompanionNordictrackTreadmill/issues/103/ @.***>

-- Roberto Viola Software engineer and open source enthusiast http://robertoviola.cloud

cagnulein commented 3 months ago

hi @victorypoint checked the 2 things:

cagnulein commented 3 months ago

for my reference image

cagnulein commented 3 months ago

@victorypoint https://github.com/cagnulein/QZCompanionNordictrackTreadmill/actions/runs/10314880819/artifacts/1793804471 i used only the last 100px of the image. it should be 10x faster now theorically, so let's see if the approach is correct

victorypoint commented 3 months ago

@cagnulein, thanks for this. I'll retest soon. Can you limit the OCR to the specific boxed areas where speed and incline values display rather than larger areas of the image? So for example, using PaddleOCR the boxes are defined as:

{'confidence': 0.9965367317199707, 'label': '15.0', 'x_min': 402, 'y_min': 1014, 'x_max': 468, 'y_max': 1048} {'confidence': 0.9980539679527283, 'label': '4.8', 'x_min': 1274, 'y_min': 1018, 'x_max': 1324, 'y_max': 1046}

cagnulein commented 3 months ago

yes it's what I would do if this approach shows improvements. also i'm using rgb888 image right now, and of course is not necesserary :)

victorypoint commented 3 months ago

@cagnulein, a bit of a problem this time with the new apk. Most of the speed changes were not transmitted to QZ. Looking at the logs it looks like the OCR may have missed many of the changes. logs.zip

cagnulein commented 3 months ago

Ok, I'll check it out. i will try to simulate on my own

Il giorno ven 9 ago 2024 alle 18:31 Al Udell @.***> ha scritto:

@cagnulein https://github.com/cagnulein, a bit of a problem this time with the new apk. Most of the speed changes were not transmitted to QZ. Looking at the logs it looks like the OCR may have missed many of the changes. logs.zip https://github.com/user-attachments/files/16564418/logs.zip

— Reply to this email directly, view it on GitHub https://github.com/cagnulein/QZCompanionNordictrackTreadmill/issues/103#issuecomment-2278324036, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAALYWB55TTKKJ6SW577HETZQTVF3AVCNFSM6AAAAABL3L5J6GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZYGMZDIMBTGY . You are receiving this because you were mentioned.Message ID: @.*** com>

victorypoint commented 3 months ago

@cagnulein, here's a logcat while I made speed and incline changes about 5 seconds apart. It looks like the OCR is picking up the values but the QZ 'changed kph' and 'changed grade' messages don't reflect all the changes. stream-logcat.zip

cagnulein commented 3 months ago

@victorypoint here i can see 1.6 , 6.4 and 8.0 speed Did you change any other speed? the coordinates seems fine because it can get it

cagnulein commented 3 months ago

got the issue! it's a simple lowercase, uppercase issue!

victorypoint commented 3 months ago

@cagnulein, here's a logcat while I made speed and incline changes about 5 seconds apart. It looks like the OCR is picking up the values but the QZ 'changed kph' and 'changed grade' messages don't reflect all the changes. stream-logcat.zip

Logcat shows "OCRlines: 5" with all the correct speed changes, but " QZ:Service: Changed KPH" messages don't show same metrics OCRlines: 5

cagnulein commented 3 months ago

fixed @victorypoint https://github.com/cagnulein/QZCompanionNordictrackTreadmill/actions/runs/10322696079/artifacts/1795843291

victorypoint commented 3 months ago

@cagnulein, good! All speed and incline changes were transmitted to QZ with no misses. Speed changes were delayed from 4-8 seconds while all incline changes delayed consistently by 4 seconds. FYI - When TM is paused (speed 0), note that the following screen is shown: 2024-08-09@14-00-44 59

cagnulein commented 3 months ago

hah ok so i can handle the text "resume". about the performance i have to investigate if it's something about grabbing the new images or the processing

cagnulein commented 3 months ago

@victorypoint last build for today (from tomorrow I will be in vacation, but I will take my Mac with me ;): I just checked the timing on my old android 8 tablet and the whole process, ocr and send it's about 500ms, so 4 seconds sounds an eternity to me. I added in this build some print on the companion to understand how the OCR process is long. If you can collect the log i can compare between my tablet to understand where is the issue https://github.com/cagnulein/QZCompanionNordictrackTreadmill/actions/runs/10323480282/artifacts/1796052301

Thanks

victorypoint commented 3 months ago

@cagnulein, with this apk there appears to be no OCR process working. See attached logcat. I confirmed that recording has been enabled on companion launch. stream-logcat.txt

victorypoint commented 3 months ago

@cagnulein, enjoy your holiday. This stuff can wait!

cagnulein commented 3 months ago

got the issue, i was trying to remove too much :D anyway from your previous log i already saw that the OCR thing is taking alone more than 1.5sec! that's crazy! so for sure i have to bring this at the minimum pieces possible. and maybe decreasing the color. i just leave you the version fixed. i'm still thinking that the best way will be having access to the usb directly

https://github.com/cagnulein/QZCompanionNordictrackTreadmill/actions/runs/10324024274/artifacts/1796194526

victorypoint commented 3 months ago

@cagnulein, if it helps I have working python code that gets a screengrab from the TM and does OCR of speed and inclination using AI.Server (on my Windows PC running PaddleOCR with nVidia card - super fast). The delay is about a second - 2 at most. The delay in this case is the TM as it does a screengrab and transfers the image to my PC over ADB.