pazaan / 600SeriesAndroidUploader

Your Medtronic 600-series pump data, direct to Nightscout
http://pazaan.github.io/600SeriesAndroidUploader/
MIT License
189 stars 311 forks source link

Non-Phone uploader only using WiFi #210

Open HenrikHL opened 6 years ago

HenrikHL commented 6 years ago

This is not really an issue with the 600SeriesAndroidUploader but a query if you have heard about a "stationary" uploader which can be wall-plugged 24/7.

My D1D daughter is now 12 and we do not monitor her a lot during the day - we only use the uploader during the night in order not to get out of our own bed in order to check the BS. The phone which we have been using so far is getting quite worn and normally only lasts for 4-6 hours (I know a new battery would solve this).

I cannot find a solution where it is possible to power the phone and at the same time have the OTG cable plugged with the Contour Next 24/7 - so I thought that someone must have tried a solution where they have an Intel Edison or RaspBerry Pi computer connected to a Contour Next? I guess having multiple of such devices could potentially cover the entire house. Only requirement is that it needs to be connected to the net via WiFi.

Any info would be appreciated :-)

And again - thumbs up to all of you for the great work - the last uploader version is fantastic!!!

Pogman commented 4 years ago

Ah did not realize the Python code had some history comms, that makes life a lot easier for you. Think I'm getting confused with different project refs.

Looks like it is missing some stuff. Some parts from here to check and maybe add to there...

https://github.com/pazaan/600SeriesAndroidUploader/blob/master/app/src/main/java/info/nightscout/android/history/PumpHistoryParser.java

https://github.com/pazaan/600SeriesAndroidUploader/blob/master/app/src/main/java/info/nightscout/android/model/medtronicNg/PumpHistoryLoop.java

https://github.com/pazaan/600SeriesAndroidUploader/blob/ae33d79ac93b203e9108edd91152d6973ed00189/app/src/main/java/info/nightscout/android/history/PumpHistoryParser.java#L339-L366

ondrej1024 commented 4 years ago

Thanks for the pointers. Incredibly complex stuff this uploader app. I wonder how many man hours went into this code. :smirk:

ondrej1024 commented 4 years ago

Hi all, I'm glad to announce that it's finally working! DD-Guard (including Nightscout uploader) is running smoothly on an RPi-Zero now. With the latest CNL driver update the E86 errors are gone and USB power cycling to recover the CNL is not needed anymore. When the RPi-Zero + CNL stick is powered with a fully charged 4400mAh power bank it runs for 20h. I might try some smaller battery which still gives me enough autonomy but is even easier to carry around.

ddguard-gw-z

wifision commented 4 years ago

Hi all, I'm glad to announce that it's finally working! DD-Guard (including Nightscout uploader) is running smoothly on an RPi-Zero now. With the latest CNL driver update the E86 errors are gone and USB power cycling to recover the CNL is not needed anymore. When the RPi-Zero + CNL stick is powered with a fully charged 4400mAh power bank it runs for 20h. I might try some smaller battery which still gives me enough autonomy but is even easier to carry around.

ddguard-gw-z

I really want this solution! Can this solution work with a continue power supply (for the night) . Where can i found the documentation ?

thanks!!!!

ondrej1024 commented 4 years ago

Hi @wifision,

the power bank can be charged connecting a wall plug power supply during the night (or whenever it is needed). The RPi-Zero will happily continue running also when charging.

You can find the documentation and source code on the DD-Guard project pages.

It would be good to have some feedback when you have tried this so I know if this works also fine for others. Thanks.

HenrikHL commented 4 years ago

Hi @ondrej1024 - does your solution also work just as a nightscout uploader - or does it require an App on your smartPhone? I just need a solution for uploading to NightScout site during the night - no need to send anything to an App...

mortlind commented 4 years ago

@ondrej1024 What CNL driver are you talking about here: "the latest CNL driver update"? I have to power cycle the USB port on the RPi every second time I poll for recent pump events.

ondrej1024 commented 4 years ago

Hi @ondrej1024 - does your solution also work just as a nightscout uploader - or does it require an App on your smartPhone? I just need a solution for uploading to NightScout site during the night - no need to send anything to an App...

Yes, you can enable only the NightScout uploader. Just leave the Blynk related parameters empty in the config file.

ondrej1024 commented 4 years ago

@ondrej1024 What CNL driver are you talking about here: "the latest CNL driver update"? I have to power cycle the USB port on the RPi every second time I poll for recent pump events.

Basically I started from the read_minimed_next24.py code by @pazaan and created cnl24driverlib.py which I use in the DD-Guard project. Initially I also needed to power cycle the USB port occasionally to recover the CNL from E86 errors.

Then I backported the communication robustness improvements by @Pogman from the Android uploader to this Python code and now it works reliable. On the RPi-Zero you cannot power cycle the USB port so robustness of the CNL driver is strictly required to make it run on this setup. It's been running here for at least 10 days now without any CNL error.

mortlind commented 4 years ago

OK. I will look into that. But could those improvements not be ported into read_minimed_next24.py? Are there other reasons for you not using, and improving, the decoding-contour-next-link code base?

wifision commented 4 years ago

Hi @wifision,

the power bank can be charged connecting a wall plug power supply during the night (or whenever it is needed). The RPi-Zero will happily continue running also when charging.

You can find the documentation and source code on the DD-Guard project pages.

It would be good to have some feedback when you have tried this so I know if this works also fine for others. Thanks.

I just bought one Raspberry Pi 4 Model B starters pack -> Now I read that the software is not compatible with older types. Is that really true >?

ondrej1024 commented 4 years ago

OK. I will look into that. But could those improvements not be ported into read_minimed_next24.py? Are there other reasons for you not using, and improving, the decoding-contour-next-link code base?

Although DD-Guard builds on parts of the project decoding-contour-next-link (namely the CNL comms driver), it is a completely separate project with different purpose and with its own GIT repository. It is not very convenient to link to code in an external repository and maintain both. So I prefer to handle everything in one place.

ondrej1024 commented 4 years ago

I just bought one Raspberry Pi 4 Model B starters pack -> Now I read that the software is not compatible with older types. Is that really true >?

What older types ? DD-Guard should run fine on any RPi model with Raspbian OS. It's just Python code, so as long as you have the Python environment and the needed libraries installed it will run.

mortlind commented 4 years ago

@ondrej1024 "So I prefer to handle everything in one place.": This exactly becomes two places, or N places, depending on how many others have this same approach, which I think is a problem. But I also agree that it always seems simpler to include a copy integrated into your project's code base Could you at least point me to the robustness improvements in your driver code, that I may port it into the https://github.com/pazaan/decoding-contour-next-link code base?

ondrej1024 commented 4 years ago

All the comms related improvements are in class Medtronic600SeriesDriver. You can just replace the complete code of that class in the original code. For details check the git history of cnl24driverlib.py (changes from 30/03/2020 and later)

mortlind commented 4 years ago

@ondrej1024 Thanks a lot!

ondrej1024 commented 4 years ago

Thanks to @oldsterIL who contributed to the CNL2.4 Python driver lib the complete data from the pump status message response is now parsed (see this commit if you are interested). So now the driver provides access to everything the pump sends in the status message. Here's an example of what we get:

### Serial ###
CNL serial: 6212-1339889

### Pump status ###
suspended:            0
bolusingNormal:       0
bolusingSquare:       0
bolusingDual:         0
deliveringInsulin:    1
tempBasalActive:      0
cgmActive:            1

### Pump alert ###
alertOnHigh:          0
alertOnLow:           0
alertBeforeHigh:      0
alertBeforeLow:       0
alertSuspend:         0
alertSuspendLow:      0
alert:                0
alertDate:            Thu Jan  1 01:00:00 1970
isAlertSilenceHigh:   0
isAlertSilenceHighLow:0
isAlertSilenceAll:    0
alertSilenceMinutesRemaining:0

### Sensor status ###
calibrating:          0
calibrationComplete:  0
exception:            0
sensorCalMinutesRemaining:   260
sensorBatteryLevelPercentage:87
sensorRateOfChange:          0.2

### Bolus ###
bolusingDelivered:        0.0
bolusingMinutesRemaining: 0
bolusingReference:        0
lastBolusAmount:          0.6
lastBolusTime:            Fri Apr 17 17:04:26 2020
lastBolusReference:       12
recentBolusWizard:        1
recentBGL:                161

### Basal ###
activeBasalPattern:       1
activeTempBasalPattern:   0
currentBasalRate:         1.0
tempBasalRate:            0.0
tempBasalPercentage:      0
tempBasalMinutesRemaining:0
basalUnitsDeliveredToday: 19.9

### Battery ###
batteryLevelPercentage:   75

### Insulin ###
insulinUnitsRemaining:    98.55
minutesOfInsulinRemaining:15555
activeInsulin:            1.6

### BGL ###
sensorBGL:                161
sensorBGLTimestamp:       2020-04-17 17:55:46.999934+02:00
trendArrow:               0

I added some of this to be displayed on the DD-Guard app:

I would like to upload these data also to Nightscout? Does anyone know if this is supported? Unfortunately the existing Nightscout API documentation is pretty useless and doesn't give me any information on this.

Pogman commented 4 years ago

NS doesn't really have a way of using that data directly except for IOB and Reservoir Units. We build a dynamic status message to include all the other info. Check this function and the related files:

https://github.com/pazaan/600SeriesAndroidUploader/blob/ae33d79ac93b203e9108edd91152d6973ed00189/app/src/main/java/info/nightscout/android/upload/nightscout/NightscoutUploadProcess.java#L454

ondrej1024 commented 4 years ago

Interesting, I'll have look. So how is this custom string displayed in NS, in the pump status pill?

Pogman commented 4 years ago

Yep

Normal state: reservoir, pump battery, time since read | transmitter battery, time to cal ns-pump-pill-normal

Pump Alert & Temp Basal in progress ns-pump-pill-alert-temp

Cal needed ns-pump-pill-calnow

Mongo DB device status entry: { "_id": { "$oid": "5e9b0da72d4ea62b4cd9e94f" }, "created_at": "2020-04-18T14:24:36Z", "device": "medtronic-600://Batman", "pump": { "battery": { "percent": 50 }, "clock": "2020-04-18T14:24:36Z", "iob": { "bolusiob": 0.9, "timestamp": "Apr 18, 2020 3:24:36 PM" }, "reservoir": 161.6, "status": { "bolusing": false, "status": "⚠ | 47% calibrate now", "suspended": false } }, "uploaderBattery": 100 }

ondrej1024 commented 4 years ago

Perfect, this helps a lot, as always. Thanks.

wifision commented 4 years ago

mimimum_data_ddguard

Hi @ondrej1024 - does your solution also work just as a nightscout uploader - or does it require an App on your smartPhone? I just need a solution for uploading to NightScout site during the night - no need to send anything to an App...

Yes, you can enable only the NightScout uploader. Just leave the Blynk related parameters empty in the config file.

It's working for a rasperry PI 4 but Only the basic values are forwarded to Nightscout. - The history data of the last update from 600SeriesAndroidUploader (pazaan) is not sending and the datapoints are not in sync

sync-problem-after-contact-600uploader-pazaan

What I do not understand that autobasal is not visible in nightscout but on the blynk's picture I can see that in the DD-guard smartphone app>?

Am I doing something wrong?

ondrej1024 commented 4 years ago

Hi @wifision

It's working for a rasperry PI 4 but Only the basic values are forwarded to Nightscout. - The history data of the last update from 600SeriesAndroidUploader (pazaan) is not sending

At the moment DD-Guard only sends live status data to NS:

History download is currently not handled by DD-Guard so no history data is sent to NS. That means you will loose the data that wasn't received by DD-Guard while the pump was out of range.

Handing of history data is not working reliably in the original CNL2.4 Python driver and I did not have enough time (and motivation) to fix that.

and the datapoints are not in sync

DD-Guards send the BGL data with the sensorBGLTimestamp read from the pump status response to the NS API. Don't know if the AndroidUploader is doing this differently. So I don't know why you see the data points slightly apart in time. Is this a problem?

What I do not understand that autobasal is not visible in nightscout but on the blynk's picture I can see that in the DD-guard smartphone app>?

In the DD-Guard smartphone app graph you see the manual boluses as they are reported by the pump (from lastBolusAmount status data) and IOB (from activeInsulin status data), no autobasal is handled.

Pogman commented 4 years ago

@ondrej1024 the pumps RTC clock drifts by 4 seconds per day and the android uploader accounts for this drift in it's event time calc at the millisecond level and is normalized to the time on the uploader device. There is always going to be some variance using multiple sources due to this. We also do checks against data already in NS that may come from multiple android uploaders to stop any dupes due to device time diffs.

ondrej1024 commented 4 years ago

Amazing, is there anything you didn't think of? :smile:

Pogman commented 4 years ago

Lol yeah I'm sure there is...

Check out the master level hacking of the Omnipod patch pump my another NS group, even includes chip decapping and scanning electron microscopy! https://blog.usejournal.com/insulin-pumps-decapped-chips-and-software-defined-radios-1be50f121d05

ondrej1024 commented 4 years ago

I read the article. This is such an incredible story. It makes you scream if you think that it would have been sooooo much easier if the insulin pump manufacturer just provided the communication protocol specs! What's so bad about being open and interoperable with other devices?

Has the CNL protocol also been reverse engineered without any support from Medtronic? I read that they collaborate with Tidepool for the Tidepool Loop integration with the new 670G. Maybe there is some possibility we get the specs for the Bluetooth enabled pump?

Pogman commented 4 years ago

All the 600 Series stuff is reversed without any additional support. The support that Tidepool is getting from Medtronic is for future pumps afaik.

ondrej1024 commented 4 years ago

Could you at least point me to the robustness improvements in your driver code, that I may port it into the https://github.com/pazaan/decoding-contour-next-link code base?

@mortlind Did you have a chance to try the CNL Python driver improvements? Any issues?

mortlind commented 4 years ago

@ondrej1024 I did make one attempt of implementing some of the changes, but it did not result in any improvement. So I will make another attempt where I will take in more of the changes. I am not giving this very high priority at the moment, since the basic decoding-contour-next-link code base works fairly well.