libfitbit compatible with "Fitbit One"?

gyhor commented 11 years ago

The Fitbit One uses bluetooth 4.0 for the communication. Is it enough to get bluetooth 4.0 working in linux for using the Fitbit One? Or does it use a completly different protocoll?

RAndrewThomas commented 11 years ago

Yep, might be some clues there. Doing "strings" on the Windows executables revealed the "logTrackerCommandBytes" setting for me.

emnullfuenf commented 11 years ago

I had a look at the logs over the last three days and did some experiments. Here are my findings so far:

Date Length/last byte Sample data (unhex) Oct 22 20:47:28 275:0 40 2 0 0 1 0 100 0 0 0 158 217 42 45 20 7 36 166 10 153 37 216 135 34 137 ... 108 233 33 12 219 165 67 235 0 0

Similarities: Byte 1-6 are always identical Byte 7 seems to increase by at least 2 over time (i wonder what happens when it reaches 255) Byte 8-16 are always identical Next-to-last byte represents dump size (<~299 = 0; <~524 = 1; <~764 = 2; <~1132 = 4; <~2244 = 8; <~4684 = 18) Last byte is always 0 Fifteen minutes without any movement are creating 267 bytes data. This seems to be the minimal size of a dump. Third-to-last byte of minimal dumps is always 231.

https://github.com/emnullfuenf/fitbit/blob/master/Sample.log

Update: Oct 23 21:18:00 283:0 40 2 0 0 1 0 252 0 0 0 158 217 42 45 20 7 242 22 40 53 168 86 69 253 209 ... 125 212 25 59 146 137 225 243 0 0 Oct 23 21:20:42 275:0 40 2 0 0 1 0 254 0 0 0 158 217 42 45 20 7 182 217 211 84 96 113 202 122 6 ... 63 164 92 188 237 207 130 235 0 0 Oct 23 21:21:41 283:0 40 2 0 0 1 0 0 1 0 0 158 217 42 45 20 7 83 153 227 54 153 163 233 99 57 ... 39 189 175 2 67 131 27 246 0 0 -> Byte 8 starts after 254 at 0 again and byte 9 gets 1

emnullfuenf commented 11 years ago

There's another documentation of Fitbit's USB protocol: https://github.com/hiptopjones/fitbit/blob/master/UsbDongle/usb_enumeration.txt

benallard commented 11 years ago

I wrote this script to analyse the logs from galileo.

I use it this way:

copy past the interesting bit from the log, ad call the script with analysedyump.py < log.txt

https://gist.github.com/benallard/7163277

emnullfuenf commented 11 years ago

I tried your script with a Fitbit Flex megadump and got this error:

m$ python analysedump.py < megadump.txt Greetings: '??R??????C' Cheering '#?F?7+VB3?' '?3?`?v;' 'W?c]:????' 66 04 6A 12 AF A6 82 EE F2 E0 2E 8B Traceback (most recent call last): File "analysedump.py", line 153, in main() File "analysedump.py", line 124, in main assert(data[index:index+4] == [0xc0, 0xdb, 0xdc, 0xdd]) AssertionError

benallard commented 11 years ago

Then it looks like our dumps are quite different. Mine is from a Fitbit One, starts with "2602", and is not ciphered.

On 26 Oct 2013, at 19:46, emnullfuenf notifications@github.com wrote:

I tried your script with a Fitbit Flex megadump and got this error:

m$ python analysedump.py < megadump.txt Greetings: '??R??????C' Cheering '#?F?7+VB3?' '?3?`?v;' 'W?c]:????' 66 04 6A 12 AF A6 82 EE F2 E0 2E 8B Traceback (most recent call last): File "analysedump.py", line 153, in main() File "analysedump.py", line 124, in main assert(data[index:index+4] == [0xc0, 0xdb, 0xdc, 0xdd]) AssertionError

— Reply to this email directly or view it on GitHub.

emnullfuenf commented 11 years ago

Interesting. Fitbit Flex starts with "2802": 2802000001005C0100009ED92A2D1407 D10665459340705A319EDEA6774F4C45 ...

ghost commented 10 years ago

I just received my Force two days ago and have started digging in a little. The megadumps from the Force also begin with "2802": 2802000001002600000051C2BE2D1908 F92AFD3260716D8BE4D9195C035C1BB5

benallard commented 10 years ago

Thanks to everyone over there, I was able to write a script to synchronise my fitbit under linux.

I didn't found a way to get the USB traces on my mac, so I had to extensively study the documentation from @sansneural, and the trace from @iluetkeb. Special thanks to you both !

The script is there: https://bitbucket.org/benallard/galileo/src/tip/galileo.py

It works for me, I download the megadump from the tracker to the fitbit server, and upload the response to the tracker back.

To be completely honest, it sometimes fails (the server complains about "INVALID_TRACKER_DATA"). I think it could be due to some conversion issue in the data format (list of integer to string to base64 string). When this happen, the next time might work just fine.

Try it, and tell me if you found some issues !

gyhor commented 10 years ago

Hi Benallard, thanks for your script. unfortunately i don't get it syncing. I tried it the last 4 days and the server always complains about INVALID_TRACKER_DATA I have a fitbit one and ubuntu 12.04

My megadump starts always with "26 02 00"

benallard commented 10 years ago

I'm sorry to read that as I have yet to understand what is causing this issue ...

At first, I thought about an encoding error, but it looks like the original Galileo daemon is using the same kind of base64 encoding (the original one with + and /). So my best guess at the moment is that I should do some kind of processing (sometimes) on the dump of the tracker before being able to upload to to the Fitbit server. But as I can't read the USB trace from my original Galileo daemon, I'm guessing by far.

As a matter of fact, I did not get the error once this week, making it even more difficult for me to dig into it ... As I said, retrying (immediately or some hours later, it did not matter) solved it for me in most of the cases.

Thanks for trying !

On 28 Nov 2013, at 12:17, gyhor notifications@github.com wrote:

Hi Benallard, thanks for your script. unfortunatly i don't get it syncing. I tried it the last 4 days and the server always complains about INVALID_TRACKER_DATA I have a fitbit one and ubuntu 12.04

— Reply to this email directly or view it on GitHub.

vincemarsters commented 10 years ago

Thanks benallard. I have it working although I have found I am getting errors (below) if I try and sync again quite quickly after a successful sync. I need to 'play' a little more to see if I can find out any requirements I need to satisfy to stop this and will post back if I find anything. To me it looks like there is a problem establishing a link to the tracker so I wonder if there is a timeout built in that needs to reset itself. If it helps I am using Ubuntu 13.10 x64 and a Fitbit Flex.

<-- 20 01 GAP_LINK_ESTABLISHED_EVENT <-- 02 07 00 (30 times) --> 03 08 01 Traceback (most recent call last): File "galileo.py", line 391, in main() File "galileo.py", line 362, in main fitbit.enableTxPipe() File "galileo.py", line 203, in enableTxPipe self.dongle.data_read(5000) File "galileo.py", line 128, in data_read raise TimeoutError main.TimeoutError

RAndrewThomas commented 10 years ago

@ benallard

I'm very pleased that you were able to create a working megadump/upload/download/minidump script working and I'm glad that the info I provided helped.

I am remiss in that I have logs of many complete transactions (timestamps on each) that I could have easily zipped up and shared to the community. I will do that soon, because it sounds like some of the comms timing built into the products may need to be followed in client-side solutions.

From my scant inspection of the logs: FitbitConnect application checks often (every 2 minutes, maybe less? don't have the logs in front of me) for a tracker. If my tracker connects, it says "no, I've updated recently" or (change of one or two bits in the reply) "yes, it's been at least 15 minutes, I'll send an update". So a tracker will connect and reply to base station requests any time (at least within whatever minute interval the base station software uses) but will refuse a dump request until ~15 minutes after the last dump, regardless of what data it may have gathered during that interval.

I do not know how the tracker may respond if the base station attempts to start a megadump after the tracker says "no, I've done that already" because the FitbitConnect application never forces a megadump if the tracker says it's up to date.

benallard commented 10 years ago

I've made few small progress, and so far those are no good news ...

I tend to believe that the data downloaded from the tracker is not 1:1 the one that should be sent to the server : definitively, the bigger the dump, the bigger the chance to get the INVALID_TRACKER_DATA error. If I synchronise every half hour / hour, it almost always go fine. If I let the data accumulate itself in the tracker, and after few days try to synchronise again, the chance is big that I will get trouble (and never recover unless I use the original galileo again). This should point at some transmission errors, but so far I have yet to see the original galileo ask for another dump because of corrupted data.

Three possibilities:

There are indeed communication errors, and I should correct the dump on the fly during download (think auto-correcting protocol), which seems overkill for such an application
There is a mandatory post-processing before uploading the dump to the server, but sometime, this post-processing doesn't need to perform anything (as it sometimes work in my case). This is quite risky as we've seen there are at least two kind of dumps: clear and encrypted.
By tweaking some parameters during the communication, I could setup the link so that what the tracker send is exactly what I should upload to the fitbit server. There I need to understand every single byte I send and receive from the tracker during connection establishment.

It would help to be able to compare the USB communication with the network communication performed by the original galileo, and see if there are differences.

RAndrewThomas commented 10 years ago

I have collected quite a few full-transfer logs with FitbitConnect (galileo?) that I could share with you, if that would help.

killua99 commented 10 years ago

I'm trying to use my fitbit one with ubuntu precise but is quite imposible I also try some vm with Win7 but without success any clue how I can see the fitbit base on my Win7 ?

benallard commented 10 years ago

I've got a good news ! I believe I just fixed the most annoying issue about the INVALID_TRACKER_DATA trouble. For the interested people, the fix is there: https://bitbucket.org/benallard/galileo/commits/c43395913ee49542ec01e8b2a5cced97ef6ed28c#Lgalileo.pyT71

Some special patterns have to be replaced in the dump between the download (from the tracker) and the upload (to the server) of the dump. I honesty don't believed that I caught all the magic dark patterns, so there might still be issues, but in my case, it is now running 100%.

Please report if this is also the case for you.

code is still here: https://bitbucket.org/benallard/galileo

benallard commented 10 years ago

Ok, I got it, the protocol used is "SLIP", which, according to Wikipedia : "SLIP is also currently used in the BlueCore Serial Protocol for communication between Bluetooth modules and host computers.[1]", which also explain those black magic with DB, DD, C0, ... And might even with its CSLIP variant help us decipher the "enciphered" dumps.

mithro commented 10 years ago

Thanks for all your work trying to decode the FitBit One protocol! Do you have any idea if the Force / Flex use the same protocol? I'd be happy to send some data dumps if that would help.

Would you be interested in working on these devices if someone was to donate you one?

hickinbottoms commented 10 years ago

Just another note of success -- with benallard's tool (bitbucket.org repo) I'm successfully syncing my One. I've now got it running as a background job on my Raspberry Pi running a flavour of Debian.

I had to update some of the timeouts in the script -- not sure whether that's just due to the slowness of the Pi or not.

Many thanks for all the hard work -- very much appreciated.

benallard commented 10 years ago

@hickinbottoms, @mitro: Thanks for the nice words!

@hickinbottoms: Feel free to share your tweaking on the timeout values, as they could benefit others. I am pretty sure there is some improvement to be done in this area, and I wouldn't blame your raspberry too quickly on this one.

@mithro: I do think they all use a similar protocol, as I've been able to spot similarities between the One and the Zip (If those both one are not completely different, chances are big that other ones are not very different either.).

killua99 commented 10 years ago

Where I can send my log? Because it's looks like galileo it's not working well for me. Done sudo service udev restart and done ... it's working! awesome guys you're awesome! :D

benallard commented 10 years ago

@killua99: I'm sorry to hear that you're having troubles with it, If you think it's not working, best you can do is open a new issue on the related bug tracker and explain there what makes you think that it's not working.

killua99 commented 10 years ago

I'll take a look to that bug tracker.

Now galileo looks like is working I can sync and so ... but need to sometime unplug and plug the USBthing to work.

rysaunders commented 10 years ago

@hickinbottoms - care to share how you've gotten the Raspberry Pi running this as a background job? May be a trivial question, but I'd love to replicate that functionality myself.

hickinbottoms commented 10 years ago

@itchytag -- I'll look into that but at the moment it's not working too well. I get lots of timeout errors so it doesn't successfully complete very often. I thought it might be a matter of just increasing the timeouts but that doesn't seem to be enough to make it work as the comms seems to stall and never restart.

As for basically getting the galileo.py script running on the pi that was pretty simple on a raspan-based system -- from memory I think I just had to install python-usb and it would run as-is, timeouts aside.

benallard commented 10 years ago

Maybe it would make sense not to try a megadump if the power of the signal is too weak, sort of like the original galileo does ...

On 14 Jan 2014, at 01:37, Stuart Hickinbottom notifications@github.com wrote:

@itchytag -- I'll look into that but at the moment it's not working too well. I get lots of timeout errors so it doesn't successfully complete very often. I thought it might be a matter of just increasing the timeouts but that doesn't seem to be enough to make it work as the comms seems to stall and never restart.

As for basically getting the galileo.py script running on the pi that was pretty simple on a raspan-based system -- from memory I think I just had to install python-usb and it would run as-is, timeouts aside.

— Reply to this email directly or view it on GitHub.

oprs commented 10 years ago

Hey there, thanks for the impressive work.

Sorry if this particular piece of information was posted before (I came across this project just a few hours ago), but I just realized that I might very well have run galileo.py before my FitBit One was even registered with FitBit Connect (but after I walked back from work with it).

So fwiw, here's a megadump from what I believe to be a brand new device. As it's not been clogged with multiple synchronization cycles yet, maybe it will help shed some light on some of the remaining dark spots: http://pastebin.com/iiHSpjNC

Oh, and here's another one from my friend's too: http://pastebin.com/pkMsBfZT (different device but also a new FitBit One, this one was dumped ~3mn later).

jonobacon commented 10 years ago

On an Ubuntu 14.04 system I am trying to install galileo and I get this:

jono@forge:~$ pip install galileo Downloading/unpacking galileo Downloading galileo-0.3.1.tar.gz Running setup.py egg_info for package galileo

Requirement already satisfied (use --upgrade to upgrade): requests in /usr/lib/python2.7/dist-packages (from galileo) Downloading/unpacking pyusb (from galileo) Could not find a version that satisfies the requirement pyusb (from galileo) (from versions: 1.0.0a2, 1.0.0a2, 1.0.0a3, 1.0.0a3, 1.0.0b1) Cleaning up... No distributions matching the version for pyusb (from galileo) Storing complete log in /home/jono/.pip/pip.log jono@forge:~$

benallard commented 10 years ago

Just created issue49 with this information. Let's continue the discussion there ...

mrquincle commented 10 years ago

I've been trying to get grips on the fitbit at https://github.com/mrquincle/fitbit-fatbat, but the encryption is a too tough nut to crack for me. What are potential ways to get it? According to https://www.ifixit.com/Teardown/Fitbit+One+Teardown/19889 there is a nRF8001 BLE chip on-board, but probably the encryption/decryption is done on the STMicroelectronics STM32L 151 ARM. It seems the STM32L151 can be read through the stlink programmer (see https://github.com/texane/stlink). This has a Cortex M3. There is quite some work out there to disassemble (IDA, Disarm, GNU), as well as decompile (desquirr). Anyone willing to get the firmware? :beers:

moyix commented 9 years ago

One thing that may possibly help others trying to reverse engineer this: if you edit /Library/Preferences/galileo_config.xml and add the line

<encryptLogs value="false" />

Detailed traces will be written to /var/log/com.fitbit.galileo.logs. Here's an example of one of these logs.

It's hard for me to tell at this point whether the communications shown are with the dongle or the actual tracker, or where the encryption/decryption is done...

openyou / libfitbit

libfitbit compatible with "Fitbit One"? #46