inonoob / pirowflo

All-in-one data interface for your Waterrower S4 Monitor or Smartrow
GNU General Public License v3.0
141 stars 32 forks source link

[BUG] SmartRow version 3 Only a few events are read from a SmartRow and then nothing else #50

Closed dan06 closed 2 years ago

dan06 commented 2 years ago

When I run PiRowFlow with input from a SmartRow and output to Ant+, only a few empty values are read from the SmartRow and then it stops. The SR seems to stay connected because the blue light does not start blinking until I stop the program. When I print out the on-row events, it looks like this is V3.00 of the SmartRow. Is that the version you tested with? Any logs I can check?

FYI I do not have an S4, but nothing in the code makes me believe it needs an S4.

Below is the console output. Any ideas?

pi@raspberrypi:~ $ /usr/bin/python3 /home/pi/pirowflo/src/waterrowerthreads.py -i sr -a 2022-02-13 14:36:09,078 - main - INFO - S4 not selected 2022-02-13 14:36:09,084 - main - INFO - inferface smartrow will be used for data input 2022-02-13 14:36:09,092 - main - INFO - Smartrow Interface started 2022-02-13 14:36:09,093 - main - INFO - Bluetooth service not used 2022-02-13 14:36:09,109 - main - INFO - Start Ant and start broadcast data start search for dongle 2022-02-13 14:36:09,502 - adapters.smartrow.smartrowreader - INFO - starting discovery Using Dynastream Innovations dongle calibration set Networkkey:b'\x00\xb9\xa5!\xfb\xbdr\xc3E' create Channel 2022-02-13 14:36:29,242 - root - INFO - found SmartRow 2022-02-13 14:36:29,246 - root - INFO - 2022-02-13 14:36:29,263 - adapters.smartrow.smartrowreader - INFO - found SmartRow macaddress 2022-02-13 14:36:31,580 - adapters.smartrow.smartrowreader - INFO - Connected to 2022-02-13 14:36:32,269 - adapters.smartrow.smartrowreader - INFO - Resolved services 2022-02-13 14:36:32,273 - adapters.smartrow.smartrowreader - INFO - Service [00001234-0000-1000-8000-00805f9b34fb] 2022-02-13 14:36:32,281 - adapters.smartrow.smartrowreader - INFO - Characteristic [00001236-0000-1000-8000-00805f9b34fb] 2022-02-13 14:36:32,288 - adapters.smartrow.smartrowreader - INFO - Characteristic [00001235-0000-1000-8000-00805f9b34fb] 2022-02-13 14:36:32,297 - adapters.smartrow.smartrowreader - INFO - Service [00001801-0000-1000-8000-00805f9b34fb] 2022-02-13 14:36:32,381 - adapters.smartrow.smartrowtobleant - INFO - SmartRow Ready and sending data to BLE and ANT Thread starting heart beat {'stroke_rate': 0, 'total_strokes': 0, 'total_distance_m': 0, 'instantaneous pace': 0, 'speed': 0, 'watts': 0, 'total_kcal': 0, 'total_kcal_hour': 0, 'total_kcal_min': 0, 'heart_rate': 0, 'elapsedtime': 0.0, 'work': 0, 'stroke_length': 0, 'force': 0, 'watts_avg': 0, 'pace_avg': 0} {'stroke_rate': 0, 'total_strokes': 0, 'total_distance_m': 0, 'instantaneous pace': 0, 'speed': 0, 'watts': 0, 'total_kcal': 0, 'total_kcal_hour': 0, 'total_kcal_min': 0, 'heart_rate': 0, 'elapsedtime': 0.0, 'work': 0, 'stroke_length': 0, 'force': 0, 'watts_avg': 0, 'pace_avg': 0} {'stroke_rate': 0, 'total_strokes': 0, 'total_distance_m': 0, 'instantaneous pace': 0, 'speed': 0, 'watts': 0, 'total_kcal': 0, 'total_kcal_hour': 0, 'total_kcal_min': 0, 'heart_rate': 0, 'elapsedtime': 0.0, 'work': 0, 'stroke_length': 0, 'force': 0, 'watts_avg': 0, 'pace_avg': 0} ^C2022-02-13 14:38:15,254 - main - INFO - Quit gracefully program has been interrupt externally - exiting

inonoob commented 2 years ago

Hey,

you could try to check what the smartrow V3.0 is sending out. You can use the app nRF Connect. You connect the smartrow with the offical app and use the nRF Connect app to snif or check the bluetooth package.

With that we can see if the new version changed something.

inonoob commented 2 years ago

Hey,

I've check again my work and I can confirm that I use the version 2.0 of the SmartRow. Which means SmartRow may have changed something. From your log, I can see that SmartRow still uses the 1234 Service with the Characteristic 1235 and 1236. 1236 is which where the data are broadcasted with a interval of 0.1 to 0.09 seconds. If the Blue led is not flashing then the PI is connected to the SmartRow. The big question now is:

Has the data output been changed ? If yes, we need to to have a log for a short rowing session. Then we can check if things have been changed.

dan06 commented 2 years ago

Thanks for the reply! I'm busy getting ready to try the nRF Connect app (if my old phone can run the SmartRow app). FYI, the SR is sending nothing after the version. The on-row handler is not being called after it sent the version number. I'm guessing it's the version because it is "V3.00". I am assuming the SmartRow device is looking for something back from the connected device.

I'll post any info I find.

dan06 commented 2 years ago

Here is a log I captured. I can't see what the server is sending to SR - I'm probably doing something wrong.

inonoob commented 2 years ago

@dan06

Smartrow v3.0 has changes the messages. The distance the information is now sort of encoded ?

I need more data please.

dan06 commented 2 years ago

If you can be more specific, I can try to provide more data.

FYI, the print statement (commented out below) does not print anything after the "V3.00" message. smartrowtobleant.py

` def on_row_event(self, event):

    #print("on_row_event: [" + str(event + "]"))

    if event[0] == self.ENERGIE_KCAL_MESSAGE:

`

inonoob commented 2 years ago

Hey again,

So I reviewed your data. And I could only see that the new SmartRow V3 has a different things I could find:

  1. the greeting message is different:

    • version 3: V@ Smartrow 'V3.00
    • version 2: V@ martrow V200'+'
  2. Distance is shifted by 10 hex numbers -version 3: 40-40-40-40-48 ==> @-@-@-@-H ==> 0-0-0-0-8 ==> 00008m (if shifted down by 10 in hex table) -version 2: 30-30-30-30-38 ==> 0-0-0-0-8 ==> 00008m but some messages for version 3 have the following: -version 3: 40-40-50-40-48 ==> @-@-P-@-H I can't understand why the SmartRow jumps in some messages to the number 50 and then goes back to 40. E.g. if distance hits 300m it will show the following 40-40-53-40-40 ==> 00300m and then jumps back to the 40-40-43-40-41 ==> 00301m.

  3. the f which was force message for the force has also been upaded: -version 3:

          66-40-40-40-47-41-2D-2D-2D-2D-2D-20-32-33-31-34-0D
    
            f   @  @  @  G  A    -     -    -     -     -     2    3   1   4  "return"

    -version 2:

            66 30 30 33 30 30 30 20 36 30 34 20 20 20 41 33 0D
    
            f      0    0  3  0    0   0        6    0  4              A  3  "return"

So what do I need, I need more data to understand:

BR

Ino

inonoob commented 2 years ago

Hey

Concerning seeing what the app is sending back to the smartrow. I myself didn't got nRF to work in order to capture the needed flag.

dan06 commented 2 years ago

I have not been able to get both sides of the conversation captured. It looks like the nRF log only captures the incoming packets, and the HCI snoop from my phone only shows the outgoing packets after I load the btsnoop in WireShark. I have two captures doing a minute of rowing with the SmartRow app. SmartRow sent different packets on startup in both cases. I tried putting both sequences into your code in place of the current reset, but neither worked.

In the WireShark capture, I don't see any lines with the source address of the SmartRow. I see only my phone, and then 'controller' and 'host'. So I don't know how to see the incoming packets interspersed with the outgoing packets. So I don't know if the app response is calculated from the incoming data from SmartRow.

At this point I don't really care about the format of the data. I just want the SmartRow device to keep sending data to the Pi. Once that is sorted out, we can look at the data format.

Below is what I see in WireShark. If this is the data we're looking for, I can say that my phone is sending the following packet sequences. Each hex number is sent as one packet. All the numbers in one line is sent in the same second, about 100ms or less, apart.

0x24, 0x23 0x24 0x0d, 0x37, 0x65, 0x66, 0x63, 0x0d 0x24, 0x24, ...

The other capture was similar, but it sent the following numbers instead of the long one above: 0x0d, 0x65, 0x36, 0x33, 0x30, 0x0d

dan06 commented 2 years ago

Some more info. I am sending data back to the rower after I read the 'V' from the rower. I get the messages logged below from smartrowtobleant.py. I am sending b'\x0d\x37\x65\x66\x63\x0d', which is exactly the events I got back. I have also not seen the keylock message before. I modified the on_row_event() as follows, just to see what is happening

` def on_row_event(self, event): print("on_row_event: [" + str(event + "]"))

    if event[0] == 'V':
        self.send_hello()

    return`

Log from python program: create Channel 2022-02-15 12:48:05,018 - root - INFO - found SmartRow 2022-02-15 12:48:05,022 - root - INFO - 2022-02-15 12:48:05,045 - adapters.smartrow.smartrowreader - INFO - found SmartRow macaddress 2022-02-15 12:48:16,451 - adapters.smartrow.smartrowreader - INFO - Connected to 2022-02-15 12:48:17,170 - adapters.smartrow.smartrowreader - INFO - Resolved services 2022-02-15 12:48:17,174 - adapters.smartrow.smartrowreader - INFO - Service [00001234-0000-1000-8000-00805f9b34fb] 2022-02-15 12:48:17,181 - adapters.smartrow.smartrowreader - INFO - Characteristic [00001236-0000-1000-8000-00805f9b34fb] 2022-02-15 12:48:17,186 - adapters.smartrow.smartrowreader - INFO - Characteristic [00001235-0000-1000-8000-00805f9b34fb] 2022-02-15 12:48:17,191 - adapters.smartrow.smartrowreader - INFO - Service [00001801-0000-1000-8000-00805f9b34fb] 2022-02-15 12:48:17,368 - adapters.smartrow.smartrowtobleant - INFO - SmartRow Ready and sending data to BLE and ANT Thread starting heart beat ]n_row_event: [ ]n_row_event: [ on_row_event: [V@] sending hello ]martRow 'V3.00' ]EYLOCK=BCFCF2D5 ]n_row_event: [ on_row_event: [7] on_row_event: [e] on_row_event: [f] on_row_event: [c] ]n_row_event: [

inonoob commented 2 years ago

Hey

so I've checked again my wireshark files for V2.00:

the handshakes goes as follows:

Pi sends in very short burst the following:

then the version 2 responds with the version number and the SmartRow has performed the reset. But in order to say connected to the smartrow every 0,1s to 0.09s the string "$" or in hex "24" must be send which is the kind of heart beat.

That's it for version 2.00.

So but you are saying that for V3.00 the handshake goes as follows ?

Pi sends in very short burst the following:

then the version 3 responds with the version number and the SmartRow.

But after that I no longer understand the handshake.

Is KEYLOCK a response ? And is "7efc" what you got back ?

BR

inonoob commented 2 years ago

Hey again,

I have the big feeling this is some sort of proof checking if this is a legit app requesting access to the SmartRow.

BR

dan06 commented 2 years ago

I agree that it looks like some challenge / response handshake.

Everything in the log after "starting heartbeat" is a response, except for "sending hello". KEYLOCK is a response. And yes, 7, e, f, c are responses (callbacks to on_row_event()). But that matches what I sent in my "hello" message.

How did you capture the data for your wireshark? As mentioned, it looks like I can capture either side of the conversation, but not both. nRF program pops up after the initial data, I think. I'll see if I can figure out how to get a complete back and forth between the old and new app.

I will clean up the logs a little to make it more clear. Obviously when it "prints" 0x0d, it is a line feed which then deletes the first character.

I'm going to pretty disappointed if I can't get this figured out. I'm not really interested in using their closed app, and only bought the SmartRow because of your project!

inonoob commented 2 years ago

Hey

So my wireshark capture are from someone which had the a nordic bluetooth sniffer. https://www.nordicsemi.com/Products/Development-tools/nrf-sniffer-for-bluetooth-le

So with that information, it was possible to find the handshake routine.

Can you check if this Keylock number has anything to do with your serial number on the device.

I have no idea why SmartRow is doing that move.

BR

dan06 commented 2 years ago

Some cleaned up logging from smartrowtobleant.py. -CR- means 0x0d.

Keylock is different every time. I am suspecting that the response is supposed to be calculated from the keylock. I have considered getting a Bluetooth sniffer device if it's not too expensive.

<-- on_row_event: [-CR-] --> RESET SmartRow <-- on_row_event: [-CR-V] <-- on_row_event: [@] <-- on_row_event: [-CR-SmartRow 'V3.00'-CR-] --> SENDING hello <-- on_row_event: [-CR-KEYLOCK=96E05CAB-CR-] <-- on_row_event: [-CR-] <-- on_row_event: [7] <-- on_row_event: [e] <-- on_row_event: [f] <-- on_row_event: [c] <-- on_row_event: [-CR-]

inonoob commented 2 years ago

Hey,

So to sum it up. The SmartRow version 3.00 sends a random 8 digit KEYLOCK to the SmartRow app. The app responses with an 4 digit response. After that the SmartRow works normal and sends some data. But we still see the problem with some sort of encoded data with the distance

BR

inonoob commented 2 years ago

Hey,

I can confirm that this is a checksum method. That means without the code from the APP. SmartRow version 3.00 do not work with PiRowFlo!

BR

dan06 commented 2 years ago

I ordered a sniffer dongle. We'll see if I can get it to work. How did you confirm it is a checksum? I am assuming that the app calculates a 4-digit hex response based on the 4-digit hex keylock value and then should send that back.

dan06 commented 2 years ago

Attaching a new log from the code that I'm playing with. I calculated a one's complement and send that to the rower. It obviously doesn't like it, and keeps sending keylock challenges.

I also realized that the response should be in ASCII hex, not binary hex, so I will fix that and play with it some more.

inonoob commented 2 years ago

Hey

So, from what I know now, it will be very hard to guess the correct response to the smartrow with trial and error.

What I can say, it might be possible to fix it so that PiRowFlo can work with version 3.00.

I' m currently following the white rabbit ;) but my skills are not that good.

inonoob commented 2 years ago

Hey

What would be very important is to have legit challenges and responses. And therefore having the Bluetooth module is also very important.

BR

dan06 commented 2 years ago

The module is on the way. I am expecting delivery on Friday, so hopefully I'll have more info after I figure out how to use it.

If it is possible to have the Pi advertise as a SmartRow in the middle, it may be possible to have the app connect to the Pi, and the Pi to SR. And have the Pi forward the challenge to the app and then the result back to the SR. That would solve two problems. We don't have to figure out the response, and we can get pass-through to the app. But that would require more development as well as making sure the app connects to the Pi and not the real SR. Just some thoughts rattling around...

Hopefully something jumps out when we see combinations of challenge / response.

inonoob commented 2 years ago

Hey

what did you mean by ASCII hex ? can you elaborate. Let's say it like this I see in some code math and some string and char manipulation. But I'm not sure about the input from the device and output from Android

BR

inonoob commented 2 years ago

Hey

Do you have at least one valid Keylock and the according response to it?

Can you also check if beside the keylock other numbers are send. Because the keylock seems not to be the only number send from the Smartrow.

You should see something like keylock followed by something. The Keylock and the something are checked against each other to check if the message has been correctly send. But that's not the response message. The key is still the keylock.

BR

BR

inonoob commented 2 years ago

Hey,

I' ve removed your adresses. If you send new logs remove them ;). You never know. You could be BANana someone's device.

dan06 commented 2 years ago

I got it! The device is sending data now. Next I will look at the data format.

inonoob commented 2 years ago

Hey

Has it something to do with multiplication and a certain division and some cuts ?

Br

dan06 commented 2 years ago

`

Has it something to do with multiplication and a certain division and some cuts ?

Yes, not necessarily in that order :) This will also be necessary if we want to implement pass through to Android. Or else pretend it's a V2.

inonoob commented 2 years ago

Hey I suppose the same path have been chosen to understand things :)

So that means, we can have in short an update for PiRowFlo in order to connect to V3.

But what I still don't understand is this distance topic like I mentioned before. Why is it 0x40 and not 0x30 for 0 for the distance and why is it switching from time to time to 0x50 in the one hundred meter range ?

but again nice job and team effort :)!

BR

inonoob commented 2 years ago

Hey @dan06,

I've checked the distance topic, it is encrypted via AND and OR of the distance input.

e.g. @ @ P B A after that manipulation will come out as 0 0 0 2 1. So here for the distance this must be checked as well.

And the two numbers before the "OD" is the Checksum of the message to check the message integrity.

BR

dan06 commented 2 years ago

I'm going through the messages one by one. So far getting stroke rate and power on my Garmin. I see that you multiply the stroke rate x2 in the existing code. From the number I get, that seems to be necessary, so I will do that also.

I ignored the manipulation you mention. I see that now - will incorporate :). I am not sure if the Garmin rower app will use distance.

My sniffer delivery has been delayed until next week so I won't be able to play with it over the weekend, but I don't think it will be necessary now.

image

inonoob commented 2 years ago

Hey

I remember why I did it. It is because for Bluetooth FTMS rowing profile you can't send float. You only can send integers. So in order to do so, you multiply the stroke rate by 2. Then any app need to divide this number by 2 to get back to the real stroke rate.

The Ant+ protocol is able to send the real stroke rate. This is why I divide back by two in the Ant+ part.

Thank for reminding me of this little detail :)

BR

inonoob commented 2 years ago

One question,

are you working on a fork which you are going to merge ?

BR

dan06 commented 2 years ago

I have not been doing things properly with Git. Once everything is working, I will create a branch and then do a pull request. Or I may just send you the one file.
I have been refactoring it a bit, but I can't test the V2 rower (which I call "legacy".) It looks like the original parsing code may work, but I chose to keep it completely separate. I have also not integrated with the elapsed-time function (yet).

I've also made a few other small changes, e.g. I changed the screen to shut down the pi, not reboot, which you may not want.

I got the distance going... Most of my time is wasted because I don't know Python that well. But I am learning quickly :)

dan06 commented 2 years ago

Looks to be mostly working for me now. However, something is still not quite 'production ready'. I had to completely restart the pi and delete and re-pair the ANT dongle to my watch. Something went wrong. I also deleted a lot of my print statements but then the communication was not good. Maybe related to having to restart. The below was captured after a power down restart and using the OLED screen to start. So that was nice!

image

inonoob commented 2 years ago

Hey @dan06,

That looks awesome!!!!

Might you be so kind and post some examples of keylocks and your results. I want to have some testing material in order to program myself a bit and also experiment.

BR

dan06 commented 2 years ago

I will generate some key response pairs.

I changed the code to call your original parsing functions since they seem to be the same as the ones I wrote. So if it's a V3, I do the challenge response and fix the distance field, and then call the existing parsing functions.

So if that all checks out, the changes to the code will not be that much. I may be able to do a pull request in the next day or two.

I have not seen any other Garmin indoor rower activity data on the internet that has all this. The power is normal Garmin power, so you get your time in power zones in GC. Garmin even allows you to set the power zones independently for cycling and other.

After I added the speed yesterday, I noticed I also get the following graphs in GC. I can also set my watch to display 500m pace. Garmin has done a really nice job with their indoor rowing app. And of course you did also by sharing this project!

image

dan06 commented 2 years ago

Both strings are started and ended with 0x0d. So on the wire it will be \rKEYLOCK=97EC60AD\r, and the response will be \ra505\r.

KEYLOCK=97EC60AD --> a505 KEYLOCK=96C85AAF --> caf5 KEYLOCK=BDA0F6C2 --> 0526 KEYLOCK=92E44BA9 --> 6f2e KEYLOCK=81D00692 --> e414 KEYLOCK=F0EBC4C3 --> 7a5b KEYLOCK=D3C550A3 --> f62f KEYLOCK=F687DCC1 --> 27bf KEYLOCK=EDDFB8DC --> 2eef KEYLOCK=EFCBBEE6 --> b857 KEYLOCK=E9A3A6B8 --> c14e KEYLOCK=92904C9A --> 7678 KEYLOCK=D0C1449F --> dadb KEYLOCK=D8AB64B8 --> df5a KEYLOCK=DA836AB6 --> f086 KEYLOCK=CEC73AC5 --> 7c31 KEYLOCK=D2B34CB1 --> 08e1 KEYLOCK=C28B0AAF --> 065d KEYLOCK=C8CF22B7 --> a5b9 KEYLOCK=B6BADACF --> 19cc KEYLOCK=B492D4A8 --> 27b0 KEYLOCK=B0D6C2B0 --> bb90 KEYLOCK=B1AEC8C3 --> ccbc KEYLOCK=9FF280AE --> 51d7 KEYLOCK=92E04DA7 --> 57ae KEYLOCK=92A44BA5 --> ee6e KEYLOCK=8DFA38BD --> 6e3e KEYLOCK=9EBE7ACC --> 1797 KEYLOCK=95965697 --> 1d47 KEYLOCK=8782208A --> 9646 KEYLOCK=89DA26AD --> a8f2 KEYLOCK=F79DDED2 --> 2c8d KEYLOCK=F4F5D2BA --> 3a4d KEYLOCK=FACDEAE3 --> 5065 KEYLOCK=FA91EAC6 --> e7b1 KEYLOCK=E8FDA2C9 --> 5d44 KEYLOCK=92C84CAC --> c720 KEYLOCK=EF99C0BF --> 0c4d KEYLOCK=E3C990AC --> 1fbc

dan06 commented 2 years ago

Something needs to be done about shutting down the connection to the smart row - not sure exactly what yet. E.g. if getting no movement for, say, 15 minutes, then close the BLE connection. Otherwise it stays connected forever if one forgets to shut down the Pi. I was hoping the SmartRow would do that, but it looks like it is expecting the client to disconnect. These are the little details to fix, once the bigger issues are sorted out :).

I think I am going to do a 10k row today instead of 10k running. It's cold outside, and I would like to give the project a real test. It will be calling the existing parsing code, so that should be a good test.

inonoob commented 2 years ago

UseHey

I'm very happy you made such good progress. It has been just 1 week until we might can confirm that smartrow version 3 will work with PiRowFlo !!!

Man thanks for your input. We might gain back and get some more people to get involved in the project.

The thing is, I only code in my spare time :). In my real life, I don't work with code at all. So I hope to inspire more people to upgrade and improve the code.

The SmartRow has also a batterie checking function which I wasn't aware. I think it is message f or e.

BR

dan06 commented 2 years ago

I have some code ready to check into a branch.

@inonoob , can you please invite me as a collaborator on your project?

After that I will create a pull request and then you can decide if you want to merge the PR to master. I have only made the minimum changes to smartrowtobleant.py.

My 10k row went great (at least from a data capture standpoint :)). Data captured in Garmin Connect looks good, and correlates well with the data I captured exactly one week ago with the SmartRow app.

dan06 commented 2 years ago

I have forked the repository and created a PR and commit on my fork. I believe you can now choose to merge this commit. I'm not used to working with forks - we create branches and then create PR from the branch back into wherever the code must go.

dan06 commented 2 years ago

I think I created a pull request for the V3 fix to your branch. Please let me know if I am doing something wrong.

DANgerous25 commented 2 years ago

@dan06 - is there any way I can get your version of the code to try whilst we wait for @inonoob to respond? I bought a pi and all the tools just to try this, all sitting idle for now. Thank you!

inonoob commented 2 years ago

Merged !