oliexdev / openScale

Open-source weight and body metrics tracker, with support for Bluetooth scales
GNU General Public License v3.0
1.7k stars 296 forks source link

Incorrect values importing from Mi scale Kg/llb #49

Closed XtractorFan closed 7 years ago

XtractorFan commented 7 years ago

Just found your app and it looks great, but.... I've been using a Mi scale since early 2016 - gave up with the MI app when I abandoned the wrist strap. At my first attempt your app synced with the scale and downloaded all the data, but the values were a multiple of x2.2 (I assume its a llb/Kg conversion?) out. The scale is set for llb's and I set your app the same. If I switch your app to Kg the values read correctly. I deleted the data and tried a re-sync, but I could only then download one value as I stood on the scale. Does downloading the data clear it from the Mi scale? and how do I correct the reading error. Thanks

oliexdev commented 7 years ago

Hi XtractorFan,

thanks for your comment. As you guessed it was interpreted as kg and so it was converted to lbs. I fixed it with commit https://github.com/oliexdev/openScale/commit/da60317da6508a29f01f7bcf435bfe3dfe2f1537

No data is deleted on the mi scale. It is impossible. It is correct that you received only new data after you received once the old data to improve the performance. Currently you have to create a new user or delete the openScale cache. With commit https://github.com/oliexdev/openScale/commit/64afdb0ff9560e56ce8f99ac0a2129e100493ca2 you can clear it under settings->measurements->delete all then you will receive again the whole mi scale history measurements.

Best regards oli

XtractorFan commented 7 years ago

Many thanks for the reply Oli, much appreciated. I'm afraid I'm not a programmer, so don't know how to compile your application - I'm limited to installing the app from Google Play which currently hosts v.1.5.2. Is there an .apk available with these changes included? In the meantime I'll try deleting the cache and re-starting with a new user. Thanks

oliexdev commented 7 years ago

I integrated to github a continuous integration process, so now you will always find the latest developer apk file in the root repository openScale-dev-build.apk

Let me hear if it works :smile:

XtractorFan commented 7 years ago

Thanks Oli, installed OK and seems to work well - I'll let you know how I get on.

XtractorFan commented 7 years ago

Hi Oli, I've just revisited your openscale and downloaded your 1.5.5 release. I'm a type 2 diabetic and I'm using a low carb (keto) diet to try and control my condition - but I'm losing too much weight..... Both myself and the wife use the Xiaomi smart scale but your app can't seem to sort our weight into 2 users - how do I do that? Myself (colin) currently weigh 142lbs and wendy weighs 123lbs - are they too close for the app to sort? Any  help would be appreciated. Thanks, XtractorFan..... On 10/06/2017 18:01:41, OliE notifications@github.com wrote: I integrated to github a continuous integration process, so now you will always find the latest developer apk file in the root repository openScale-dev-build.apk [https://github.com/oliexdev/openScale/blob/master/openScale-dev-build.apk] Let me hear if it works 😄 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub [https://github.com/oliexdev/openScale/issues/49#issuecomment-307577567], or mute the thread [https://github.com/notifications/unsubscribe-auth/Ab7MFpAVrm8BoAgmDb811EL4bk1otdOeks5sCsv0gaJpZM4N0KS8].

oliexdev commented 6 years ago

@XtractorFan Please create two users with your initial weight. After that enable under preferences the "smart user assignment". On the next synchronisation the weight should be assign to the user which has the latest nearest weight.

XtractorFan commented 6 years ago

Hi Oli, Tried that several time - clearing data and cache each time to no avail, all data comes in (when it works) on the currently displayed user - my wife's data doesn't get updated. Also the bluetooth comms are very erratic, showing 'lost connection' frequently - I read your note, so I know you had to 'reverse engineer' the comms with the Mi scale - but it's so hit-and-miss.... I have tried it on my Moto G£ (works most of the time), Samsung Tab S2 and Lenovo P2 - mostly can't persuade them to download data at all. I can export/send you the .csv if you have any suggestions. Have you found a better scale? - if so anything you could recommend? Thanks again, XtractorFan

------ Original Message ------ From: "OliE" notifications@github.com To: "oliexdev/openScale" openScale@noreply.github.com Cc: "XtractorFan" colin@lbhfarm.co.uk; "Mention" mention@noreply.github.com Sent: 30/10/2017 07:31:53 Subject: Re: [oliexdev/openScale] Incorrect values importing from Mi scale Kg/llb (#49)

@XtractorFan https://github.com/xtractorfan Please create two users with your initial weight. After that enable under preferences the "smart user assignment". On the next synchronisation the weight should be assign the user which has the latest nearest weight.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/oliexdev/openScale/issues/49#issuecomment-340364058, or mute the thread https://github.com/notifications/unsubscribe-auth/Ab7MFuZg4POFhZjoM0PzdPVdLGBTFM8-ks5sxXtpgaJpZM4N0KS8.

oliexdev commented 6 years ago

hmm... I think I found a bug in openScale but I am not sure. Can you test it again but please set your initial weight in kg not in lbs because I forgot to convert the initial weight into the internal basis weight which is kg.

Do you sync to the scale always with different smartphones or tablets? Does your connection is also lost if you only sync with one device?

XtractorFan commented 6 years ago

Yes, a bug.... My scale is set to lbs, but your app only seems to work in kg. This is what happens when I add the user's using lbs and the 'target' at Colin 143lb and Wendy 123lb: it doesn’t sort the data into 2 Colin & Wendy bins....

But if I set it to kg AND set the target to 143kg 123kg, the data comes in and gets sorted into the 2 user bins: but of course the results are in kg, not lbs - no conversion takes place

I cannot change the output from the Mi, as the only other option is Chinese!

As for the BT comms - as I've said before I have 2 years of data in the scale and so far I have never succeeded in downloading all the data. Only the Motorola phone works consistently, but even that stops and starts and 'freezes' - even though I keep briefly stepping on the Mi Scale to try and keep it awake. If I forcibly stop your app, next time it runs I see it has downloaded some more data, but not the complete range. So I never know if the comms are continuing, if it's stopped or what's going on - some incrementing display of reading data input would be helpful - so you can see if things are working?

Hope this helps, Cheers, XtractorFan

------ Original Message ------ From: "OliE" notifications@github.com To: "oliexdev/openScale" openScale@noreply.github.com Cc: "XtractorFan" colin@lbhfarm.co.uk; "Mention" mention@noreply.github.com Sent: 31/10/2017 12:41:13 Subject: Re: [oliexdev/openScale] Incorrect values importing from Mi scale Kg/llb (#49)

hmm... I think I found a bug in openScale but I am not sure. Can you test it again but please set your initial weight in kg not in lbs because I forgot to convert the initial weight into the internal basis weight which is kg.

Do you sync to the scale always with different smartphones or tablets? Does your connection is also lost if you only sync with one device?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/oliexdev/openScale/issues/49#issuecomment-340750156, or mute the thread https://github.com/notifications/unsubscribe-auth/Ab7MFlQmBpZcFMICbVq6zt9X9PsqHNbJks5sxxVpgaJpZM4N0KS8.

oliexdev commented 6 years ago

My scale is set to lbs, but your app only seems to work in kg. This is what happens when I add the user's using lbs and the 'target' at Colin 143lb and Wendy 123lb: it doesn’t sort the data into 2 Colin & Wendy bins....

yeah as I guessed it was my fault I didn't convert the initial weight to the user scale unit. It should be fixed now. Please install and test the beta version of openScale, see https://github.com/oliexdev/openScale/releases. Enter your initial weight in your selected unit with the beta version.

As for the BT comms - as I've said before I have 2 years of data in the scale and so far I have never succeeded in downloading all the data. Only the Motorola phone works consistently, but even that stops and starts and 'freezes' - even though I keep briefly stepping on the Mi Scale to try and keep it awake. If I forcibly stop your app, next time it runs I see it has downloaded some more data, but not the complete range. So I never know if the comms are continuing, if it's stopped or what's going on - some incrementing display of reading data input would be helpful - so you can see if things are working?

As far as I know the mi scale doesn't delete any data on the scale. So after 2 years you probably have hundreds of entries in the internal RAM of the mi scale. Every time you delete all measurements under Settings->Measurements all history data are send to openScale. This will take some time but you should see the receiving data as toast texts. It is important to receive all data because in the end openScale acknowledge to the mi scale that you received the history data. Next time if you sync with the mi scale only new history data are send since the last sync. If you cancel the initial history transmission no acknowledge will be send and openScale will be always try to receive all history data again. In my opinion it is a design mistake of the company to do that but I can't change this behaviour. A good question is what happened with the scale if the internal RAM is full? Will the scale still work? I don't know.

XtractorFan commented 6 years ago

Hi Oli, thanks for that - a big improvement, but still a few issues.... Wendy's data still has a few out of range values, looks like it has wrongly sorted them from mine to her bin. Wendy doesn't weigh herself every day, but I do. My data has July, August and September data missing, but it is dumped into her readings.

Also the charting doesn't work for 2017 - see above. The comms now seem better - you are right, it takes a while to download all the data, but it now came in OK on the Samsung Tab S2 (but still with the above problem). The 'toast text' is erratic, looks like it slows down the comms enough for the app to catchup with the data coming in - perhaps? Could I export the data, re-sort it manually then re-import it to get it correct, or have you any other ideas?

A big improvement, many thanks for your efforts! Regards, XtractorFan

------ Original Message ------ From: "OliE" notifications@github.com To: "oliexdev/openScale" openScale@noreply.github.com Cc: "XtractorFan" colin@lbhfarm.co.uk; "Mention" mention@noreply.github.com Sent: 31/10/2017 18:39:30 Subject: Re: [oliexdev/openScale] Incorrect values importing from Mi scale Kg/llb (#49)

My scale is set to lbs, but your app only seems to work in kg. This is what happens when I add the user's using lbs and the 'target' at Colin 143lb and Wendy 123lb: it doesn’t sort the data into 2 Colin & Wendy bins....

yeah as I guessed it was my fault I didn't convert the initial weight to the user scale unit. It should be fixed now. Please install and test the beta version of openScale, see https://github.com/oliexdev/openScale/releases. Enter your initial weight in your selected unit with the beta version.

As for the BT comms - as I've said before I have 2 years of data in the scale and so far I have never succeeded in downloading all the data. Only the Motorola phone works consistently, but even that stops and starts and 'freezes' - even though I keep briefly stepping on the Mi Scale to try and keep it awake. If I forcibly stop your app, next time it runs I see it has downloaded some more data, but not the complete range. So I never know if the comms are continuing, if it's stopped or what's going on - some incrementing display of reading data input would be helpful - so you can see if things are working?

As far as I know the mi scale doesn't delete any data on the scale. So after 2 years you probably have hundreds of entries in the internal RAM of the mi scale. Every time you delete all measurements under Settings->Measurements all history data are send to openScale. This will take some time but you should see the receiving data as toast texts. It is important to receive all data because in the end openScale acknowledge to the mi scale that you received the history data. Next time if you sync with the mi scale only new history data are send since the last sync. If you cancel the initial history transmission no acknowledge will be send and openScale will be always try to receive all history data again. In my opinion it is a design mistake of the company to do that but I can't change this behaviour. A good question is what happened with the scale if the internal RAM is full? Will the scale still work? I don't know.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/oliexdev/openScale/issues/49#issuecomment-340861980, or mute the thread https://github.com/notifications/unsubscribe-auth/Ab7MFrC3szhk7x17RRxkoYtbgd4aCY_Eks5sx2lhgaJpZM4N0KS8.

oliexdev commented 6 years ago

hmm... I don't know why some of your data are mixed up? Did your two weights overlap for a period of time? No the toast text doesn't slow down the app. It is only that you receive faster data then the toast text "display time". I don't understand what you mean with re-sort? The sorting in the app depends on the weight date. You can't change it. But you can export your data and your wife data in two separate csv files. Then you can copy your mixed data between these two files and import it again in a clean openScale environment. Note: The export/import function only export/import all data for the selected user!

XtractorFan commented 6 years ago

Hi Oli, hope they didn't overlap - that would mean myself or Wendy putting on/losing 20lbs I guess it's probably a 'glitch' in the data stored in the scale, I'm sure it's not your app - as it happens on both tablets. Something has improved the data download function, as it works on the Samsung now - great! Yes, that's what I meant by re-sort, export the 2 data sets, swap the suspect months then import it back in again - sorted.....

By a 'clean openScale environment' do you mean just deleting the user data?

What will happen to new data from the Mi scale? - will it try and re-download the whole data again - or will it have told the Mi scale it has all the data? I'll give it a whirl and see what happens Thanks again for your help, now I can concentrate on tracking and not losing any more weight! Regards, XtractorFan

------ Original Message ------ From: "OliE" notifications@github.com To: "oliexdev/openScale" openScale@noreply.github.com Cc: "XtractorFan" colin@lbhfarm.co.uk; "Mention" mention@noreply.github.com Sent: 01/11/2017 17:34:03 Subject: Re: [oliexdev/openScale] Incorrect values importing from Mi scale Kg/llb (#49)

hmm... I don't know why some of your data are mixed up? Did your two weights overlap for a period of time? No the toast text doesn't slow down the app. It is only that you receive faster data then the toast text "display time". I don't understand what you mean with re-sort? The sorting in the app depends on the weight date. You can't change it. But you can export your data and your wife data in two separate csv files. Then you can copy your mixed data between these two files and import it again in a clean openScale environment. Note: The export/import function only export/import all data for the selected user!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/oliexdev/openScale/issues/49#issuecomment-341179865, or mute the thread https://github.com/notifications/unsubscribe-auth/Ab7MFt-pazCb_6CoIhZh371kAV1QeXNjks5syKuLgaJpZM4N0KS8.

oliexdev commented 6 years ago

First of all I am sorry that you have so much trouble with openScale but you seem to be a "power user" :wink: So I am appreciate for your help to improve openScale. :+1:

I am not sure if its really a glitch because the user assignment is clear programmed: "Assign to the user with the nearest last weight value...." :thinking:

Yeah with clean openScale environment I mean delete the whole user data or reinstall the openScale and add two new users and then import the csv file regarding to the users.

If you remove the openScale user and add a new one yeah openScale will try to re-download the whole data again. But if you already imported the data via the csv data no data is really imported from the scale because measurements with the same weighting time/date are duplicated and are not imported!

For your information I implemented an option to estimation calculation for body water and fat percentage, so your mi scale will be in the next release a more "smart" scale. :wink:

頑張ってください

XtractorFan commented 6 years ago

No reason to be sorry Oli, just glad I can help improve your very useful program. I look forward to your next update. Regards, Colin

------ Original Message ------ From: "OliE" notifications@github.com To: "oliexdev/openScale" openScale@noreply.github.com Cc: "XtractorFan" colin@lbhfarm.co.uk; "Mention" mention@noreply.github.com Sent: 01/11/2017 20:02:41 Subject: Re: [oliexdev/openScale] Incorrect values importing from Mi scale Kg/llb (#49)

First of all I am sorry that you have so much trouble with openScale but you seem to be a "power user" 😉 So I am appreciate for your help to improve openScale. 👍

I am not sure if its really a glitch because the user assignment is clear programmed: "Assign to the user with the nearest last weight value...." 🤔

Yeah with clean openScale environment I mean delete the whole user data or reinstall the openScale and add two new users and then import the csv file regarding to the users.

If you remove the openScale user and add a new one yeah openScale will try to re-download the whole data again. But if you already imported the data via the csv data no data is really imported from the scale because measurements with the same weighting date are duplicated and not imported!

For your information I implemented an option to estimation calculation for body water and fat percentage, so your mi scale will be in the next release a more "smart" scale. 😉

頑張ってください

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/oliexdev/openScale/issues/49#issuecomment-341223254, or mute the thread https://github.com/notifications/unsubscribe-auth/Ab7MFva9wqsHSvuXiNAr83dINzyaNgWzks5syM5ggaJpZM4N0KS8.