search

Tigge / antfs-cli

Extracts FIT files from ANT-FS based sport watches such as Garmin Forerunner 60, 405CX, 310XT, 610 and 910XT.
MIT License
312 stars 76 forks source link

Vivofit support #99

Closed psukys closed 9 years ago

psukys commented 10 years ago

It seems to be partly working with vivofit. The workout first workout file is downloaded the right way, although I'm not sure about the second (1989 yr.) file Here's a paste of the log, maybe I should just add an exception for this file? http://pastebin.com/KhfSjHWG

Tigge commented 10 years ago

Oh, a new device. Is it this one http://sites.garmin.com/vivo/? This bug might already be fixed on the split branch, https://github.com/Tigge/Garmin-Forerunner-610-Extractor/tree/split, could you try it with this branch? It requires the https://github.com/Tigge/openant library.

psukys commented 10 years ago

Yes, it is vivofit :) I tried this branch and it retrieved current data without problems. I'll be testing it out throughout the week and post my findings :)

Tigge commented 10 years ago

Excellent. Keep me posted!

psukys commented 10 years ago

It seems that antfs-cli downloads only empty file of current time, even though I had an activity on (using the windowish Garmin connector after, it synced today's and activity's file. http://pastebin.com/r6x9q9kT

psukys commented 10 years ago

I'm not sure, but might there be a problem that uploading and downloading files are mixed up?

mart-e commented 10 years ago

Hello,

I have tried with the split branch and same as for the master branch, I just get a 274 bytes file

Tigge commented 10 years ago

I wonder if the same bug as getting weight data is present here. If all data are in the same file but that file is updated with new data it will not be downloaded again. What sort of data are one expected to get from the vivofit? It might also be that it is not in a category recognized by antfs-cli. Can someone post a log file?

mart-e commented 10 years ago

Ok after doing some debugging of the client, it seems that the files are present but it's just not the correct ones that are uploaded.

If I execute directory.print_list(), I get the following result

Index   Type    FIT Type    FIT Number  Size    Date    FIT Flags   Flags
1   1   14  0   64512   1989-12-31 01:00:00     0   -w----
2   128     2   0   200     1989-12-31 01:00:00     0   -w----
3   128     11  0   200     1989-12-31 01:00:00     0   -w----
4   128     3   0   200     1989-12-31 01:00:00     0   -w----
5   128     1   0   274     2014-07-05 22:01:00     0   r-----
6   128     32  36  865     2014-06-29 17:34:00     0   r-----
7   128     32  37  346     2014-06-29 22:01:00     0   r-----
8   128     32  38  342     2014-06-29 22:02:00     0   r-----
9   128     32  39  350     2014-06-29 22:03:00     0   r-----
10  128     32  40  377     2014-06-29 22:06:00     0   r-----
...
65  128     32  100     306     2014-07-04 10:12:00     0   r-----

So it seems that the vivofit uses the type 32 to record steps and activities. Adding 32 in the Identifier makes the software download all these files. I can now upload it on the manual import page on the garmin website but I am not sure it got all my steps (displays 3000 for yesterday while I got at least three time that). I guess it's because I get several files for the same day and it remembers only the first one.

I will make a pull request to add the 32 type. @Tigge do you prefer I do it on the master of this repository or on the openant ?

update: ok I don't get it, it seems that the following days, I did not get any files following the first synchronisation

update 2: so it seems the vivofit cleans the memory after sync with my phone but I miss the "update your directory now" command

Tigge commented 10 years ago

Pull requests on openant is very much prefered. Thanks for that. Both the openant and antfs-cli pull request are now merged.

So it only counts steps since last download? A bit weird. Right now antfs-cli will not redownload your file if it has changed. A quick fix to redownload if the archived flag is not set. This will work if you are always connecting the device to the same computer. A real fix will take a bit longer.

mart-e commented 10 years ago

Thanks @Tigge for the merges. After further investigations, I figured out that I need one "empty sync" (meaning I put the vivofit in sync mode but my laptop is not) to force the device to generate .fit file. At the next sync (with my laptop this time), I get a fresh .fit file at the time of the empty sync. Weird but working...

minybot commented 10 years ago

I success to download some fit files from vivofit. But those fit files are corrupted.

-- 1117. #55: unknown (6 entries) -------------------------- unknown [uint32]: 774853680 unknown [uint32]: 0 unknown [uint32]: 586000 unknown [uint16]: 279 unknown [uint16]: 788 unknown [enum]: 0

-- 1118. #55: unknown (6 entries) -------------------------- unknown [uint32]: 774853680 unknown [uint32]: 3540 unknown [uint32]: 784000 unknown [uint16]: 505 unknown [uint16]: 788 unknown [enum]: 6

-- 1119. #125: unknown (3 entries) ------------------------- unknown [uint32]: 774853680 unknown [byte]: unknown [byte]: �� ��

Traceback (most recent call last): File "./sample_program.py", line 141, in a.parse(hook_func=print_hook_func) File "/home/ukcco/research/sport/python-fitparse/fitparse/activity.py", line 9, in parse raise FitParseError("File parsed is not an activity file.") fitparse.exceptions.FitParseError: File parsed is not an activity file.

Tigge commented 10 years ago

@mart-e that is a bit weird. Did anyone try my suggestion with the archived flag? It requires some python knowledge but in antfs-cli.py:

   downloading[filetype] = filter(lambda fil: self.get_filename(fil) not in local_files[filetype], remote_files[filetype])

should be something like

   downloading[filetype] = filter(lambda fil: self.get_filename(fil) not in local_files[filetype] or not fil.is_archived(), remote_files[filetype])
Tigge commented 10 years ago

@minybot the files downloaded from Vivofit are probably not FIT Activity files but rather other types of FIT files that contains information about heartbeat rate and such.

minybot commented 10 years ago

Is any way or document to extract the information from vivofit Fit files? Btw, I can download all files from my vivofit successfully.

mart-e commented 10 years ago

I will try the archive flag this weekend. @minybot I guess you are referring to the fitparse library. Try with the ng branch on it. The fitdump script give you some output but I haven't figured out the meaning yet. But as said @Tigge, the .fit files are a bit different (steps every x minutes and heartbeat mixed)

Tigge commented 10 years ago

If you is comfortable with the licence agreement, there is a FIT SDK available here http://www.thisisant.com/resources/fit with all the information you'll ever need. I'm guessing the interesting data from the Vivofit is in the monitoring_a/b files (id 15 and 32) or perhaps daily totals (id 28)

On 25 July 2014 11:16, Martin Trigaux notifications@github.com wrote:

I will try the archive flag this weekend. @minybot https://github.com/minybot I guess you are referring to the fitparse https://github.com/dtcooper/python-fitparse library. Try with the ng branch on it. The fitdump https://github.com/dtcooper/python-fitparse/blob/ng/scripts/fitdump script give you some output but I haven't figured out the meaning yet.

— Reply to this email directly or view it on GitHub https://github.com/Tigge/Garmin-Forerunner-610-Extractor/issues/99#issuecomment-50126358 .

minybot commented 10 years ago

From fitdump

  1. file_id
    • garmin_product: 1837
    • manufacturer: garmin
    • number: 27
    • serial_number: 3883044730
    • time_created: 2014-07-20 16:00:00
    • type: 32
    • unknown_6: 0
  2. device_info
    • battery_voltage: 5.94921875 [V]
    • hardware_version: 1
    • manufacturer: garmin
    • product: 1837
    • serial_number: 3883044730
    • software_version: 2.8
    • timestamp: 2014-07-25 16:27:00
  3. monitoring_info
    • local_timestamp: 2014-07-21 02:00:00
    • timestamp: 2014-07-20 16:00:00
    • unknown_1: (6, 1)
    • unknown_3: (7962, 11943)
    • unknown_4: (326, 1028)
    • unknown_5: 2175
    • unknown_7: (7171, 7171)
  4. monitoring
    • unknown_26: 39809
    • unknown_27: 69

  5. monitoring

    • unknown_26: 39824

    * unknown_27: 70

  6. monitoring
    • unknown_24: (104,)
    • unknown_26: 52828
  7. monitoring
    • unknown_24: (8,)
    • unknown_26: 52888
  8. monitoring
    • unknown_24: (72,)
    • unknown_26: 52948
  9. monitoring
    • unknown_24: (8,)
    • unknown_26: 54808
  10. monitoring
    • unknown_24: (72,)
    • unknown_26: 54928
  11. monitoring
    • unknown_24: (8,)
    • unknown_26: 56008
  12. monitoring
    • unknown_24: (72,)
    • unknown_26: 56128
  13. monitoring
    • unknown_24: (8,)
    • unknown_26: 56548

  14. monitoring

    • unknown_24: (104,)

    * unknown_26: 56608

  15. monitoring

    • cycles: 0 [cycles]
    • unknown_19: 6
    • unknown_24: (0,)

    * unknown_26: 57508

I think this is easy to extract now, unknown_26: time unknown_27: heart beat number

unknown_24: I am not sure

minybot commented 10 years ago
  1. monitoring
    • unknown_26: 4867
    • unknown_27: 76
  2. event
    • data: 1
    • data16: 1
    • event: 41
    • event_type: marker
    • timestamp: 2014-07-21 00:30:00
  3. monitoring
    • unknown_24: (72,)
    • unknown_26: 4872
  4. monitoring
    • active_time: 262.0 [s]
    • activity_type: generic
    • cycles: 0 [cycles]
    • timestamp: 2014-07-21 00:30:00
    • unknown_19: 132
    • unknown_29: 510
  5. monitoring
    • active_time: 437.0 [s]
    • activity_type: walking
    • cycles: 1332 [cycles]
    • timestamp: 2014-07-21 00:30:00
    • unknown_19: 206
    • unknown_29: 510

  6. unknown_125

    • unknown_1: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
    • unknown_2: (127, 0, 0, 0, 381, 0, 2, 199, 42, 0, 0, 0, 0, 0, 0, 30149)

    * unknown_253: 774837000

    end of file

Tigge commented 9 years ago

Hopefully this will work better with #120. Closing. Re-open if it still fails.

balu- commented 9 years ago

seems like #120 (or 32aa009ab45a566a2c3aa16412756630ea64415b in particular) make antfs-cli redownload every single .fit file on the watch on every snyc?

Not familiar with python code, i only observed the behavior of the script, and by reverting the commit i got the old behavoir where only new files are transmitted. Is there a reason to transfer all the files everytime?

Tigge commented 9 years ago

There should be no change in behavior with this commit. If it doesn't work anymore there is probably a bug in there somewhere. I'm not able to reproduce this on my system.

balu- commented 9 years ago

my bad redownload is triggerd by this change (as the commit message says :flushed: ) https://github.com/Tigge/antfs-cli/commit/da76e054ea1618a805c4e34789479dfd35021929