pcolby / bipolar

Free your data from Polar FlowSync
GNU General Public License v3.0
115 stars 16 forks source link

how can I help #2

Closed thetrimaster closed 10 years ago

thetrimaster commented 10 years ago

Sorry I'm not a programmer but have the V800, RS800, RCX5, CS600, CS500 for testing. Would be great if you find a way for transfer.

pcolby commented 10 years ago

Thanks @thetrimaster. I've got some data to play with now, but hopefully will need some more very soon.

Do you, by any chance, have any V800 sensors other than the HR7? eg bike and/or foot sensors?

Do you feel like installing the hook now? That way, when we have figured out how to parse the protobuf data, you'll have some captured data to try it on.

Thanks again.

thetrimaster commented 10 years ago

Hello Paul,

Bernhard here.

First, nice to get in contact with you and I’m very intersted to help.

At the moment I have the footpod sensor for the V800, and also the Wahoo bike speed/cadence sensor which usually works but Sometimes just get weired and I have to reset it (do not really trust this model - monday built?? )

But very soon maybe already this week I will get also the original Polar speed and the cadence sensors.

If you tell me how to install the hook (don’t know som much what it is and what it does) I surely can do it and test things which are of interest. As I have all those different units I hope we can figure out how things playing together.

Kind regards

Bernhard

Skickat från Windows E-post

Från: Paul Colby Skickat: ‎måndag‎ den ‎30‎ ‎juni‎ ‎2014 ‎10‎:‎18 Till: pcolby/bipolar Kopia: Bernhard Tri

Thanks @thetrimaster. I've got some data to play with now, but hopefully will need some more very soon.

Do you, by any chance, have any V800 sensors other than the HR7? eg bike and/or foot sensors?

Do you feel like installing the hook now? That way, when we have figured out how to parse the protobuf data, you'll have some captured data to try it on.

Thanks again.

— Reply to this email directly or view it on GitHub.

pcolby commented 10 years ago

The hook (more details at https://github.com/pcolby/bipolar/wiki/Hooks) is a small replacement file you drop into Polar FlowSync, and once in place, it logs all training session data that Polar FlowSync uploads to their website.

To install the hook, first, you need to locate your Polar FlowSync installation - usually its installed in C:\Program Files (x86)\Polar\Polar FlowSync.

In that directory, you'll see a file called Qt5Network.dll - this is a networking library from the open-source Qt project. We're going to replace it with a slightly modified version.

So, copy (or move) the original Qt5Network.dll file to somewhere safe. Personally, I just rename if to Qt5Network.dll.backup.

Next, download the new Qt5Network.dll from github. You can check https://github.com/pcolby/bipolar/releases for the latest version - currently its https://github.com/pcolby/bipolar/releases/download/v0.0.1/Qt5Network.dll

Once you've downloaded the DLL, drop it into the Polar FlowSync directory.

Finally, restart Flow Sync (if its already running) - right-click the FlowSync systray icon, click "Close". Click "Start", type "FlowSync" into the box, and click on FlowSync.

Done.

Now, when you next sync a training session, you should see binary files appearing under C:\User\<your login>\AppData\Local\Polar\PolarFlowSync\export.

If you want me to expand on any of the above, just let me know.

Cheers.

PS If you're concerned, or just curious, about what code I've added to this DLL, you can read the diff at: https://github.com/pcolby/bipolar/blob/master/hook/qt/qnetworkaccessmanager.patch

thetrimaster commented 10 years ago

wow … very exciting. Will test tomorrow. But Im here in Sweden and it is late in the evening.

Was watching the game of my team Germany. Which finally ended well. 😉

Tack to you tomorrow.

Kind regards

Bernhard

Skickat från Windows E-post

Från: Paul Colby Skickat: ‎tisdag‎ den ‎1‎ ‎juli‎ ‎2014 ‎00‎:‎26 Till: pcolby/bipolar Kopia: Bernhard Tri

The hook (more details at https://github.com/pcolby/bipolar/wiki/Hooks) is a small replacement file you drop into Polar FlowSync, and once in place, it logs all training session data that Polar FlowSync uploads to their website.

To install the hook, first, you need to locate your Polar FlowSync installation - usually its installed in C:\Program Files (x86)\Polar\Polar FlowSync.

In that directory, you'll see a file called Qt5Network.dll - this is a networking library from the open-source Qt project. We're going to replace it with a slightly modified version.

So, copy (or move) the original Qt5Network.dll file to somewhere safe. Personally, I just rename if to Qt5Network.dll.backup.

Next, download the new Qt5Network.dll from github. You can check https://github.com/pcolby/bipolar/releases for the latest version - currently its https://github.com/pcolby/bipolar/releases/download/v0.0.1/Qt5Network.dll

Once you've downloaded the DLL, drop it into the Polar FlowSync directory.

Finally, restart Flow Sync (if its already running) - right-click the FlowSync systray icon, click "Close". Click "Start", type "FlowSync" into the box, and click on FlowSync.

Done.

Now, when you next sync a training session, you should see binary files appearing under C:\User\AppData\Local\Polar\PolarFlowSync\export.

If you want me to expand on any of the above, just let me know.

Cheers.

PS If you're concerned, or just curious, about what code I've added to this DLL, you can read the diff at: https://github.com/pcolby/bipolar/blob/master/hook/qt/qnetworkaccessmanager.patch

— Reply to this email directly or view it on GitHub.

pcolby commented 10 years ago

I'd be very interesting in seeing a captured route file (one of the ones ending in *-route) where the route was an almost-straight line on a continuous incline or decline; eg riding / running / walking up or down a long fairly straight hill.

I'll probably be able to record such a route myself on the weekend, but if anyone has one in the meantime, and doesn't mind sharing, attach one here :smile:

(Want to test my guess that tag "4" is altitude data...)

Thanks.

thetrimaster commented 10 years ago

Paul,

I will check what I have … or does it only work with future files?

Second question: In the case you got a sync working for export of files … would it also be possible for a simple sync-app for windows phone 8? or is this something totally different?

Kind regards

Bernhard

Skickat från Windows E-post

Från: Paul Colby Skickat: ‎tisdag‎ den ‎1‎ ‎juli‎ ‎2014 ‎14‎:‎05 Till: pcolby/bipolar Kopia: Bernhard Tri

I'd be very interesting in seeing a captured route file (one of the ones ending in *-route) where the route was an almost-straight line on a continuous incline or decline; eg riding / running / walking up or down a long fairly straight hill.

I'll probably be able to record such a route myself on the weekend, but if anyone has one in the meantime, and doesn't mind sharing, attach one here :smile:

(Want to test my guess that tag "4" is altitude data...)

Thanks.

— Reply to this email directly or view it on GitHub.

pcolby commented 10 years ago

I will check what I have … or does it only work with future files?

Only works on future files I'm afraid. The hook works by observing network requests as they happen, so it won't see the data for any sessions synced prior to the hook installation.

That's one of the reasons why its helpful to get the hook DLL in place now... it means that when there is an app ready to convert the captured data, there'll (hopefully) be a nice, extensive set of data to test against :smile:

Second question: In the case you got a sync working for export of files … would it also be possible for a simple sync-app for windows phone 8? or is this something totally different?

Interesting question. It depends or whether or not Polar used Qt for their WP8 app... while its technically possible, I would guess that they didn't.

I don't have a Windows Phone, but assuming you can view files on the device, have a look at the Polar app's directory, and see what files are there... are there any that start with "Qt"? If so, it should be possible to make both the hook and the final app work on WP8 (with some effort, of course). If not, then it would be something entirely different - we'd probably need a C# expert for that :wink:

pcolby commented 10 years ago

Awesome news!! I've worked out how the latitude, longitude, and altitude data is encoded! :smile:

thetrimaster commented 10 years ago

Paul,

I see you are much faster than I even can get going and read your updates. (A lot of work at the moment and also some coaching - backlog as usual)

Congrates that you already found out some coding … nice.

About Windows phone 8 … the problem is there is absolutely nothing in Place for Windows Phone. The only thing I can do is to see flow on the phone´s web browser. ☹

Skickat från Windows E-post

Från: Paul Colby Skickat: ‎onsdag‎ den ‎2‎ ‎juli‎ ‎2014 ‎09‎:‎22 Till: pcolby/bipolar Kopia: Bernhard Tri

I will check what I have … or does it only work with future files?

Only works on future files I'm afraid. The hook works by observing network requests as they happen, so it won't see the data for any sessions synced prior to the hook installation.

That's one of the reasons why its helpful to get the hook DLL in place now... it means that when there is an app ready to convert the captured data, there'll (hopefully) be a nice, extensive set of data to test against :smile:

Second question: In the case you got a sync working for export of files … would it also be possible for a simple sync-app for windows phone 8? or is this something totally different?

Interesting question. It depends or whether or not Polar used Qt for their WP8 app... while its technically possible, I would guess that they didn't.

I don't have a Windows Phone, but assuming you can view files on the device, have a look at the Polar app's directory, and see what files are there... are there any that start with "Qt"? If so, it should be possible to make both the hook and the final app work on WP8 (with some effort, of course). If not, then it would be something entirely different - we'd probably need a C# expert for that :wink:

— Reply to this email directly or view it on GitHub.

pcolby commented 10 years ago

I see you are much faster than I even can get going and read your updates

All depends on how busy every other aspect of life is at the time... :sweat_smile:

(A lot of work at the moment and also some coaching - backlog as usual)

I know what that's like!

About Windows phone 8 … the problem is there is absolutely nothing in Place for Windows Phone. The only thing I can do is to see flow on the phone´s web browser. ☹

I see. Yeah, in that case, WP8 will be something completely different, unless Polar get around to releasing a Qt-based WP8 app. I haven't worked on MS phones since the Windows Mobile SmartPhone days.

thetrimaster commented 10 years ago

hi Paul,

finally I did the stuff and now got a bunch of files in the folder you mentioned. Let me know if you need any for further investigation.

Btw. how can those files be readed with some editor?

Skickat från Windows E-post

Från: Paul Colby Skickat: ‎tisdag‎ den ‎1‎ ‎juli‎ ‎2014 ‎00‎:‎26 Till: pcolby/bipolar Kopia: Bernhard Tri

The hook (more details at https://github.com/pcolby/bipolar/wiki/Hooks) is a small replacement file you drop into Polar FlowSync, and once in place, it logs all training session data that Polar FlowSync uploads to their website.

To install the hook, first, you need to locate your Polar FlowSync installation - usually its installed in C:\Program Files (x86)\Polar\Polar FlowSync.

In that directory, you'll see a file called Qt5Network.dll - this is a networking library from the open-source Qt project. We're going to replace it with a slightly modified version.

So, copy (or move) the original Qt5Network.dll file to somewhere safe. Personally, I just rename if to Qt5Network.dll.backup.

Next, download the new Qt5Network.dll from github. You can check https://github.com/pcolby/bipolar/releases for the latest version - currently its https://github.com/pcolby/bipolar/releases/download/v0.0.1/Qt5Network.dll

Once you've downloaded the DLL, drop it into the Polar FlowSync directory.

Finally, restart Flow Sync (if its already running) - right-click the FlowSync systray icon, click "Close". Click "Start", type "FlowSync" into the box, and click on FlowSync.

Done.

Now, when you next sync a training session, you should see binary files appearing under C:\User\AppData\Local\Polar\PolarFlowSync\export.

If you want me to expand on any of the above, just let me know.

Cheers.

PS If you're concerned, or just curious, about what code I've added to this DLL, you can read the diff at: https://github.com/pcolby/bipolar/blob/master/hook/qt/qnetworkaccessmanager.patch

— Reply to this email directly or view it on GitHub.

pcolby commented 10 years ago

finally I did the stuff and now got a bunch of files in the folder you mentioned.

Great :)

Let me know if you need any for further investigation.

Will do.

Btw. how can those files be readed with some editor?

Obviously I'm working on an app to convert them. In the meantime, the best you can do (and it's not great) is push them through the protoc program, which is part of the Google Protocol Buffers suite.

Note though, while you can download protoc for Windows, you will also need to have gunzip installed (the FlowSync comms are compressed), which you can get lots of ways (I use the version included with Git for Windows).

So, assuming you download and extract both protoc.exe and gunzip.exe, you can do something like:

gunzip -c exported-prolar-proto-file | protoc --decode_raw

For one of my route sessions, it outputs something like:

9 {
  1 {
    1: 2014
    2: 6
    3: 29
  }
  2 {
    1: 5
    2: 5
    3: 14
    4: 0
  }
  3: 1
}
2: "<binary string representing latitude data>"
3: "<binary string representing longitude data>"
4: "<binary string representing GPS altitude data>"
5: "<binary string representing the number of satellites data>"
1: "<binary string representing time / duration offsets>"
2: ....
3: ....
etc

Have a read of https://github.com/pcolby/bipolar/wiki/Polar-API-V2 and/or the proto files in issue #3 for some explanantion of what fields are what.

Cheers.

pcolby commented 10 years ago

Just FYI, I've created a Bipolar group on Google Groups - http://groups.google.com/d/forum/bipolar-app

Feel free to continue the conversation here or there - whichever is easiest for you :smile:

Cheers.

thetrimaster commented 10 years ago

would like to avoid to go deeper into Google … 😉

btw … any news … anything I can help etc.

Regards

Bernhard

Skickat från Windows E-post

Från: Paul Colby Skickat: ‎torsdag‎ den ‎17‎ ‎juli‎ ‎2014 ‎04‎:‎16 Till: pcolby/bipolar Kopia: Bernhard Tri

Just FYI, I've created a Bipolar group on Google Groups - http://groups.google.com/d/forum/bipolar-app

Feel free to continue the conversation here or there - whichever is easiest for you :smile:

Cheers.

— Reply to this email directly or view it on GitHub.

pcolby commented 10 years ago

No probs.

There's been a lot of progress on the parsing side of things. All of the protobuf stuff is working, and GPX is working now as well (all via unit tests only currently, but that will change soon). I've had to release an updated Windows hook DLL to make TCX conversion possible - that was released earlier today.

I've also begun working on the NSIS installer, but the main focus right now is on TCX conversion, which is coming along nicely.

Hopefully I'll have a first GUI-less version complete with Windows installer sometime next week... but as always, it all depends on how much time I manage to find in between everything else :smile:

I don't have any cadence data yet, and I will need some for testing soon. So, if you're willing to provide the data files from a sample run and/or ride with foot pod / bike cadence sensor(s), that would be really helpful. Otherwise I'll just have to wait for my own sensors (which I plan on ordering in the next few days).

thetrimaster commented 10 years ago

Hi Paul,

as I have the program installed as adviced by you and also doing all workouts (either with footpod or cadence on the bike … the data should be there … question is only how to identify ... or tell me which files you want and I can send them.

As there is a IM coming soon I have data for almost every day 😉

Cheers

Bernhard

Skickat från Windows E-post

Från: Paul Colby Skickat: ‎fredag‎ den ‎18‎ ‎juli‎ ‎2014 ‎14‎:‎37 Till: pcolby/bipolar Kopia: Bernhard Tri

No probs.

There's been a lot of progress on the parsing side of things. All of the protobuf stuff is working, and GPX is working now as well (all via unit tests only currently, but that will change soon). I've had to release an updated Windows hook DLL to make TCX conversion possible - that was released earlier today.

I've also begun working on the NSIS installer, but the main focus right now is on TCX conversion, which is coming along nicely.

Hopefully I'll have a first GUI-less version complete with Windows installer sometime next week... but as always, it all depends on how much time I manage to find in between everything else :smile:

I don't have any cadence data yet, and I will need some for testing soon. So, if you're willing to provide the data files from a sample run and/or ride with foot pod / bike cadence sensor(s), that would be really helpful. Otherwise I'll just have to wait for my own sensors (which I plan on ordering in the next few days).

— Reply to this email directly or view it on GitHub.

pcolby commented 10 years ago

as I have the program installed as adviced by you and also doing all workouts (either with footpod or cadence on the bike … the data should be there … question is only how to identify ... or tell me which files you want and I can send them

Great! The cadence and stride data is both included in the one file - the ones that end ing *-samples.

There's also some cadence and stride info included in the `*-laps' files if you take any manual laps.

So ideally, give us a couple of recent files endings in -samples, and there's non-zero *.laps files with the same name, provide them too :D

You can just drag them onto a new comment here, or email them to git at colby dot id dot au. Also, let me know if you're willing for me to add those files to the Bipolar test suite in the github repo (not necessary, but preferred).

PS the files should be somewhere like: C:\User\<your login>\AppData\Local\Polar\PolarFlowSync\export\v2-<your id>-training-sessions-<session id>-exercises-<exercise id>-samples. You can match the <session id> number against the URLs you get in flow.polar.com, such as "19946380" in https://flow.polar.com/training/analysis/19946380, to see which sessions the data files are for.

Cheers.

pcolby commented 10 years ago

There's been no activity on this issue for more than 30 days, and since I've just released the first 0.3.x milestone release, I'll close this issue now.

Free free to create new issues for anything new that comes up.

Thanks! :smile: