pazaan / 600SeriesAndroidUploader

Your Medtronic 600-series pump data, direct to Nightscout
http://pazaan.github.io/600SeriesAndroidUploader/
MIT License
189 stars 312 forks source link

Mine The Gap - Approximate SGV when calibration is missed #229

Closed Pogman closed 5 years ago

Pogman commented 6 years ago

An unfortunate feature of the 600 pumps is the hard cut-off when a calibration is missed.

While the pump will not report a SGV we can access the ISIG and previous calibration data.

Using a Kalman filter running on the ISIG history and a algorithm (K{ISIG} - OFFSET) * FACTOR produces very high quality SGV value that is mostly bang on target or within a point or two of the pumps own calc.

This would be a 'experimental setting' that has to be engaged via the 'Advanced System Settings' with an appropriate warning of it's approximate nature.

whooze commented 6 years ago

Is it possible to add this data as RAW or something in that case? To make it obvious in Nightscout that it's not an actual cgm-value from the pump?

Is there ISIG values during the warmup-period? If that stuff could be posted as RAW too it would be awesome. Sensors tend to fail when we really don't want them to, having RAW these hours would for sure be a bonus

Pogman commented 6 years ago

I have looking into using NS raw and white dot indicators for this but unfortunately it is completely unusable as it totally depends on all sorts of very specific Dexcom stuff.

I have been able to find a way of indicating when a approximation is in use:

ns-test-approx-symbol

This symbol does not show for any other 600 event or sgv type that we use.

The user will also be getting "calibrate now" notice and some sort of "approx. sg" warning.

Pogman commented 6 years ago

I've also looked at the Warm-Up period and it is way too unreliable to approximate and at best you might gain 30 - 60 minutes and need several finger bg's to use.

The Medtronic ISIG value is all we get and it's meaningless without calibration data.

Even with a sensor restart the way sensors work produces inconsistent data until they stabilise. Part of the reason for this is that during the warm-up procedure the sensor is blasted with a higher voltage.

Pogman commented 6 years ago

Some preliminary test results...

ALL SGV APPROXIMATED:

ns-test-aprox

NORMAL PUMP SGV:

ns-test-real

Pogman commented 6 years ago

This mostly done and just being doing a live test over the last 2 days. It's surprisingly weird not having the tyranny of 12 hour calibrations and being able to wait for the best time to calibrate or just not worrying about it if I'm out without my bg meter. This will also estimate past the sensor end-of-life.

May look into adding a 'estimation quality' indicator based on the historical stability of calibrations and amount of time since the cal was missed. It seems very reliable indeed when a sensor is relatively stable.

Hard to quantify my personal results with general use and there may be issues with the 670G.

EG from the uploader with the estimated period in blue running from the missed cal to the next cal.

ns-uploader-est-sgv-cal

cfaagaard commented 5 years ago

You could "misuse" the json to show the details in nightscout. bg_test

Json: { "_id": { "$oid": "5bbb15492bdb6bb36e3259ce" }, "enteredBy": "cfa", "eventType": "Calculated SGV", "glucose": 7, "reason": "", "glucoseType": "ISIG: 35,33", "duration": 0, "units": "mmol", "created_at": "2018-10-08T07:45:00.000Z" }

Have not tested to see how it effects reports, xdrip and other things….. :-)

Pogman commented 5 years ago

Naughty ;)

Yeah I would be wary of doing this to a main NS element for the things you noted and any future changes to NS.

Ideally at some point specific PR-ed mods to NS that support some of the 600 uploader features would be nice.

cfaagaard commented 5 years ago

he he.... I know. Another json thing; did you know about the foodtype in the treatment-json and the support for unicode characters? I am testing it to see if it could be of any use. Example: { "_id": { "$oid": "5bba6cbb02aa6d8d35df1735" }, "carbs": 40, "enteredBy": "test", "timestamp": "2018-10-08T05:29:16Z", "foodType": "☯ test of foodtype", "created_at": "2018-10-08T05:29:16Z", "eventType": "Meal Bolus", "absorptionTime": 120 } Notice foodtype: the yin yang sign is unicode: \u262F and then the text "test of foodtype". In NS it looks like this: foodtype The Loop-guys use it to show different pictures for different foodgroups.

there is a lot to choose from: https://www.unicode.org/charts/

Pogman commented 5 years ago

The problem with unicode is that different devices tend to co-op them for their own use or a user has installed a character set that changes their appearance significantly. I've had to remove some that were being used by the uploader due to this.

th122 commented 5 years ago

Not using the Enlite much at the moment (some of those marvellous people from our community have been supplying me with G5 sensors and transmitters), but I absolutely love this development. Been giving the WIP Arecibo branch a try in parallel, and it's marvellous to see how much better the Enlite gets when it doesn't keep disrupting the circadian rythm so terribly.

You're working magic, and it's making all the difference in using the 640G as a BG source!

tuzoenduro commented 5 years ago

Hello all, hello Pogman, any idea when this as well as the "CGM Source" choice for the uploader will be delivered? I am only waiting on my openaps rig to start trying this out and I intend to use the 640 as a source of BG for the moment (have not been able to find a Minilink transmitter yet) so I would be interested to know when I can use this and not have to modify files or write cron scripts for this to work.

thanks for all your phenomenal work, and happy new year!!

Pogman commented 5 years ago

I need to find some time for the next 'push' but you can find WIP code in my repo. As is it's at the around 90% phase to beta and should be quite useable but you will need to compile it yourself.

What was the 'CGM Source' thing? You can just turn off the pump history component in the uploader and it will only deal with the cgm sending it to NS and/or xDrip.

tuzoenduro commented 5 years ago

Ahh, cool, although I'm a little afraid of the whole "compiling myself" thing. I will read how to carry this out thanks!

For the CGM, I have been reading through tons of back issues to get ready to start looping and I read an issue ticket from Lennart regarding being able to chose if you wanted to export only CGM values or CGM + pump.. I think it was ticketed for the next version but I could have miss-read and this being something already available. The thing is I cannot find the issue again so I cannot confirm.

On Thu, Jan 3, 2019 at 12:47 PM Pogman notifications@github.com wrote:

I need to find some time for the next 'push' but you can find WIP code in my repo. As is it's at the around 90% phase to beta and should be quite useable but you will need to compile it yourself.

What was the 'CGM Source' thing? You can just turn off the pump history component in the uploader and it will only deal with the cgm sending it to NS and/or xDrip.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pazaan/600SeriesAndroidUploader/issues/229#issuecomment-451121664, or mute the thread https://github.com/notifications/unsubscribe-auth/APr1vJtNsEtMJzvvhkzXFFGcqmuJoxjEks5u_e3RgaJpZM4V5KCe .

--


Dr. Luis Toussaint

Pogman commented 5 years ago

Ah yeah I sort of remember that... The uploader will send device status info for the pump to NS and not just cgm and maybe this was to have an option to disable so cgm and absolutely nothing else.

tuzoenduro commented 5 years ago

Yep, that is the point. I have seen through the net people either manually disabling the info on their NS sites or using CRON scripts to remove the items, but from what I read Lennart was planning on implementing this directly, this is why I was asking about the next version being available.

On Thu, Jan 3, 2019 at 1:03 PM Pogman notifications@github.com wrote:

Ah yeah I sort or remember that... The uploader will send device status info for the pump to NS and not just cgm and maybe this was to have an option to disable so cgm and absolutely nothing else.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pazaan/600SeriesAndroidUploader/issues/229#issuecomment-451124894, or mute the thread https://github.com/notifications/unsubscribe-auth/APr1vM4GCRd7PDCQ_HuF6v_-qTj0g5ryks5u_fGngaJpZM4V5KCe .

--


Dr. Luis Toussaint

Pogman commented 5 years ago

OK there is no work done for that feature currently. If Lennart doesn't have time for it I will look into doing it for the beta.

tuzoenduro commented 5 years ago

Ok, thanks!! For me this is more doable than finding transmitters, as they are either expensive or very hard to find. So this would be a good solution and a lot cleaner than running scripts to clean un NS.

Dr. luis toussaint


From: Pogman notifications@github.com Sent: Thursday, January 3, 2019 1:11:14 PM To: pazaan/600SeriesAndroidUploader Cc: tuzoenduro; Comment Subject: Re: [pazaan/600SeriesAndroidUploader] Mine The Gap - Approximate SGV when calibration is missed (#229)

OK there is no work done for that feature currently. If Lennart doesn't have time for it I will look into doing it for the beta.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/pazaan/600SeriesAndroidUploader/issues/229#issuecomment-451126395, or mute the threadhttps://github.com/notifications/unsubscribe-auth/APr1vJcMvaSxORdOTYpVZ2BQI1nwcaD0ks5u_fNigaJpZM4V5KCe.

tuzoenduro commented 5 years ago

I had found this on a google group for looping

"Hi Tomás,

it took me a while to dig through my archives, and find the line in question, but here it is: To stop OpenAPS and the uploader from battling for the validity of their data in nightscout, you need to comment out (or delete) the line reading "uploadStatus(statusRecords, uploaderBatteryLevel);" from the file "app/src/main/java/info/nightscout/android/upload/nightscout/NightScoutUpload.java". The source to the current release (indra, 0.6.1) is located at "https://github.com/pazaan/600SeriesAndroidUploader".

see also @pogman in https://gitter.im/pazaan/decoding-contour-next-link?at=5a8ac6cfc3c5f8b90dd77499
(The effects taking place otherwise are described in the posts leading to that hint)

PS: There's a very promising WIP branch of an hopefully upcoming release at "https://github.com/Pogman/640gAndroidUploader" which is even able to interpolate sgv approximations from ISIG values while the pump itself is witholding these to enforce calibration - I found it very helpful to be able to delay calibration to times of steady BG without having to continuously monitor the raw isig myself. While experimenting with an untethered loop that might be an interesting branch to look at.

best regards, Tatjana"

I don't know if this could help you.. Also, it talks about this same github thread...

pazaan commented 5 years ago

@tuzoenduro is the issue you mentioned previously https://github.com/pazaan/600SeriesAndroidUploader/issues/201?

tuzoenduro commented 5 years ago

YES!!!

Thank you Lennart, that is the issue I was talking about. I think I skipped it while looking for it, or got thrown off by the IOB pill on the title and decided that was not it...

On Thu, Jan 3, 2019 at 1:40 PM Lennart Goedhart notifications@github.com wrote:

@tuzoenduro https://github.com/tuzoenduro is the issue you mentioned previously #201 https://github.com/pazaan/600SeriesAndroidUploader/issues/201?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pazaan/600SeriesAndroidUploader/issues/229#issuecomment-451132086, or mute the thread https://github.com/notifications/unsubscribe-auth/APr1vPDF2K22dcBnWEhKI9Culu3a_P8pks5u_foigaJpZM4V5KCe .

--


Dr. Luis Toussaint

Pogman commented 5 years ago

Closed - completed and available in 0.7.0 release