search

Dolnor / mifit-data-export

Set of Unix tools to grab data from Mi Fit Android app, most of this is courtesy of xmxm
75 stars 10 forks source link

Problems to extract data in non-root phone #6

Open vicmanmo opened 7 years ago

vicmanmo commented 7 years ago

Hi,

I need to extract MyBand data to create a virtual sensor in OpenIoT.

I have a non-rooted Xioami RedMi Note 2 and I set ForceBackupMode value to Y in run.sh, but in my phone I don't see backup screen. In the other project of author is @xmxm, I see backup screen, but the program doesn't run.

Can you help me?

The result of running run.sh is this:

extraction started on 02/23/2017 09:36 cannot find database files. non-rooted phone? attempting native backup approach adb server is out of date. killing...

Thank you very much

vicmanmo commented 7 years ago

I have changed run.sh for using adb of my OS, and now I see backup screen and read registers.

But tar: apps/com.xiaomi.hm.health/db/origin_db: it doesn't find the file.

Can you help me?

The result of running run.sh is this:

extraction started on 02/23/2017 10:27 cannot find database files. non-rooted phone? attempting native backup approach press Backup My Data button on device... adb server is out of date. killing...

Thank you very much

vicmanmo commented 7 years ago

When it executed

adb backup -f mi.ab -noapk -noshared com.xiaomi.hm.health

The file mi.ab doesn't have data.

Where is the problem??

jreznik commented 6 years ago

Same here - it doesn't matter if I use system adb or recent platform-tools adb from Google. com.xiaomi.hm.health is even not in the full backup.

ErikBjare commented 5 years ago

Same, tried to figure out why but no luck

ironjan commented 5 years ago

Hi,

the mi fit app now has an export functionality, most like because of GDPR. Through some google search I came across the URL the in-app browser of Mi Fit would be using for the export: https://user.huami.com/hm_account/2.0.0/index.html?loginPlatform=web&platform_app=com.xiaomi.hm.health&v=s3.1.1#/

You can get your data from there*. I just checked the files and the numbers matched the app so far. The export contained 6 CSV files in my case:

(NAME OF FILE).csv
(CSV HEADER)

ACTIVITY_<ID>.csv
date,lastSyncTime,steps,distance,runDistance,calories

BODY_<ID>.csv
timestamp,weight,height,bmi,fatRate,bodyWaterRate,boneMass,metabolism,muscleRate,visceralFat,impedance

HEARTRATE_<ID>.csv
date,lastSyncTime,heartRate,timestamp

SLEEP_<ID>.csv
date,lastSyncTime,deepSleepTime,shallowSleepTime,wakeTime,start,stop

SPORT_<ID>.csv
type,startTime,sportTime,distance,maxPace,minPace,avgPace,calories

USER_<ID>.csv
userId,gender,height,weight,nickName,avatar,birthday

*I did not test the app export functionality itself as it requested me to login again - with my main Google account in an app-internal browser, despite using Google as my "Mi Fit App login".

ErikBjare commented 5 years ago

@ironjan That's great news! Trying it out now, thanks a bunch for letting us know :heart:

Edit: Export worked fine, but disappointed to only find day-resolution data and not minute resolution for steps/heartrate :disappointed: Was that available in the exports made with the mifit-data-export tool?

ironjan commented 5 years ago

TL;DR: GDPR complaint caused export to gain minute-wise data.

@ErikBjare Took a while to come back. After your comment I checked the data and verified that it is not minute-wise. Since then I have requested minute-wise data from xiaomi (my mistake; even though the play store privacy agreement states Xiaomi, the data seems to be at Huami). After there was no reaction, I issued a GDPR complaint to the data protection office of Northrhine-Westphalia and received word a few weeks back that "the app has been updated to export minute-wise data". This was untrue then; I tried again today and the most export contained minute wise data. I did not yet verify if the export is complete.

Extract: 

$ ls
ACTIVITY.csv  ACTIVITY_MINUTE.csv  ACTIVITY_STAGE.csv  BODY.csv  HEARTRATE_AUTO.csv  HEARTRATE.csv  SLEEP.csv  SPORT.csv  USER.csv
$ head ACTIVITY_MINUTE.csv
date,time,steps
2018-03-24,18:26,12
2018-03-24,18:29,104
2018-03-24,18:32,117
2018-03-24,18:33,37
2018-03-24,18:34,104
2018-03-24,18:37,127
2018-03-24,18:39,116
2018-03-24,18:40,115
2018-03-24,18:41,112
$ head HEARTRATE.csv
date,lastSyncTime,heartRate,timestamp
1565020753,70
1564605323,76
1554157496,72
1546406148,62
1545737025,81
1545006196,80
1542865300,78
1542788260,72
1542615574,67
$ head HEARTRATE_AUTO.csv
date,time,heartRate
2018-04-12,19:48,47
2018-04-12,19:50,53
2018-04-12,19:52,65
2018-04-12,19:54,53
2018-04-13,01:52,74
2018-04-13,01:54,68
2018-04-13,01:56,71
2018-04-13,01:58,69
2018-04-13,02:00,70
ErikBjare commented 5 years ago

@ironjan Amazing! Thank you so much for putting in the effort!

howff commented 2 years ago

The new website address is https://mifit.huami.com/t/account_mifit (which, for me, redirects to https://user.huami.com/privacy2/index.html?loginPlatform=web&platform_app=com.xiaomi.hm.health&v=4.0.17#/)

However it still does not include all of the data which is in the app. For example sleep data, it only has total minutes in each zone, whereas the app has exact start times of each zone transition. Does anyone know how to get huami to update their export with this data?