nightscout / minimed-connect-to-nightscout

Community edition of original
MIT License
46 stars 138 forks source link

mmconnect and 770g in US #11

Open gotpancreas opened 3 years ago

gotpancreas commented 3 years ago

Not sure whether this is the right place to post (I’m new to GitHub), but I’ve been trying to set up a new NS instance to connect with my 770g via CareLink and the MiniMed Mobile app. I’ve got NS working fine, but there mmconnect portion seems to be returning a 404 error and I’m not able to see any SGV or pump data in NS. I’ve confirmed that the data appears in CareLink as expected and that my CareLink credentials are correct in mmconnect as well. The login page for CareLink US has recently changed, so I’m wondering if it uses a different server than before (or a problem similar to this)? I’ve read discussion about issues with CareLink EU as well, but not so much about US. I’m happy to provide logs, do testing, or whatever I’m capable of to help this project and make it work!

Thanks for all this community has done so far, and thanks in advance for your help!

justincrosby commented 3 years ago

Hey all, I've been scouring the internet for a solution and I think this project might be the one. I'm basically just looking for a way to view my CGM data on my wear OS watch. I have a 770g and the minimed mobile app working but I don't see any way of forwarding to the watch unfortunately. As others have suggested ideally I just want to access the persistent notification data on my watch. I'm a firmware developer in my day job so please let me know if there's any area I can help in. At the very least I can help with testing :)

benceszasz commented 3 years ago

@FredMK unfortunately MiniMed Mobile uses custom views: image

benceszasz commented 3 years ago

Although the online CareLink Connect web app outside the US does not support 7xxG pumps however the blePereodicDataEndpoint in country settings also works outside the US for 780G. This means, that the last 24 hours of 7xxG pumps can be accessed the same way outside US as @bewest described in his earlier post. Of course there are some differences in case of 780G pump data (for example the auto correction in treatments), but the structure of the JSON data is the same. Of course a care partner account is required outside the US too, but it seems logical: the patient uses the MiniMed Mobile app, while the care partner uses the CareLink Connect app and only the latter requires access to the CareLink Cloud.

ondrej1024 commented 3 years ago

@benceszasz thanks for the info. However I don't really understand what this means.

Do we have all the information needed to write the code which pulls the 7xxG data from the the CareLink server, both US and Europe? Would it be possible to provide an API description ?

For me it is still not exactly clear how authentication works and how the data can be requested from CarelLink.

benceszasz commented 3 years ago

CareLink uses OAuth since June 2020, which was implemented in minimed-connect-to-nightscout: https://github.com/nightscout/minimed-connect-to-nightscout/pull/2

I have also used the same method in the CareLink Follower xDrip+ datasource although I still haven't implemented the refresh token (I always do a new login, when the old token is not working anymore): https://github.com/NightscoutFoundation/xDrip/pull/1649

The difference with the new devices is that the data of the last 24 hours is not provided anymore by the /patient/connect/data endpoint of CareLink MiniMed servers, but by the blePereodicDataEndpoint in the country settings response of the /patient/countries/settings CareLink MiniMed endpoint. The information for pulling the US data was already provided by Ben in his previous comments: https://github.com/nightscout/minimed-connect-to-nightscout/issues/11#issuecomment-774228928 https://github.com/nightscout/minimed-connect-to-nightscout/issues/11#issuecomment-774511043 This pull logic is based on the US version of the online CareLink Connect webapp, which works with 770G pumps.

The last open question was just how it works outside the US, because the online CareLink Connect webapp doesn't support the new pumps, it still uses the old /patient/connect/data endpoint. It turned out, that blePereodicDataEndpoint information is also provided by the /patient/countries/settings endpoint outside the US too and it also works: the endpoint provides the data of the last 24 hours of the new 7xxG pump models.

ondrej1024 commented 3 years ago

Thanks a lot for the explanation. I will dig into it myself and try to put all the bits and pieces together.

benceszasz commented 3 years ago

I have extended the xDrip CareLink Follower datasource to support the 7xxG pumps too in addition to Guardian Connect CGMs: https://github.com/benceszasz/xDripCareLinkFollower

I have also extracted the CareLink communication layer from the follower datasource in a separate repository: https://github.com/benceszasz/CareLinkJavaClient

ondrej1024 commented 3 years ago

I have also extracted the CareLink communication layer from the follower datasource in a separate repository: https://github.com/benceszasz/CareLinkJavaClient

That's great. Just what I was looking for. And it's actually working for me (780G in Italy). :smile:

benceszasz commented 3 years ago

In normal cases both the modified xDrip CareLink Follower and the standalone CareLink client should work with Guardian Connect CGMs, 770G and 780G pumps, but there can be special currently not handled cases most likely with 7XXG pumps and maybe also with GC inside US. The original xDrip CareLink follower was tested by many Guardian Connect users in Hungary, so it seems to be stable and that is why I have created a pull request for xDrip. The modifications required by 7xxG pumps are new and the data is much more complex than the simple CGM data of GC.

iwanvandaele commented 3 years ago

Very nice, thx!!! Just tested in with a 780G pump in Belgium and it's working fine with xdrip. I'm very happy!!!

ClifClimber commented 3 years ago

Hi .. very nice news ! ! ! .. I've tested in a 780G in Spain and is working ! ! ! The only bug is that sometimes does not differentiate between smbs and treatments :

image

Great job ! ! ! !

bewest commented 3 years ago

Excellent. Thanks, this is phenomenal work, @benceszasz.

hendrytl commented 3 years ago

@ClifClimber currently I am trying to create an alternative solution for downloading data from CareLink instead of MMConnect, which is currently seems to be blocked by Medtronic when used in Heroku. My original scope was Guardian Connect, but I am working on extending it to other devices too. Because it is not MMConnect I think it is not related to this thread. I will write you in Gitter.

Does this mean that mmconnect would work if deployed in Azure instead of Heroku? I'm in the US with 770G.

benceszasz commented 3 years ago

@hendrytl maybe mmconnect will be able to connect to CareLink, I don't know if Medtronic also blocks Azure or just Heroku. On the other hand mmconnect currently doesn't work with the new pumps (7xxG). If you have an Android device (doesn't have to be the phone connected to the pump, but any Android device), you can try the xDripCareLinkFollower with which you can use xDrip to follow 7xxG pumps or upload to Nightscout using Nightscout REST API sync function of xDrip+

ondrej1024 commented 3 years ago

It turned out, that blePereodicDataEndpoint information is also provided by the /patient/countries/settings endpoint outside the US too and it also works: the endpoint provides the data of the last 24 hours of the new 7xxG pump models.

@benceszasz could you please provide the exact format or a sample data request to blePereodicDataEndpoint ? When I try this here I get a 400 response, so it looks like I am not formatting the request correctly.

I see in your Java code that you send user and role information as JSON formatted data in the request but you set "Content-Type" to "application/x-www-form-urlencoded" in the header. This confuses me. :thinking:

ondrej1024 commented 3 years ago

I don't understand. What I am doing wrong?

image

benceszasz commented 3 years ago

@ondrej1024 the role seems to be wrong, it should be carepartner

There is a mapping for the role in the CareLinkCloud blePereodicDataEndpoint, see the getUserRole() in User class: https://github.com/benceszasz/CareLinkJavaClient/blob/master/src/main/java/info/nightscout/medtronic/carelink/message/User.java

Region (OUS or not) doesn't seem to be relevant in CareLink Cloud, only the core role type (patient or carepartner).

ondrej1024 commented 3 years ago

Great. Now the last piece fell into place. I had missed the mapping of the role types. Thanks a lot.

ondrej1024 commented 3 years ago

I have also extracted the CareLink communication layer from the follower datasource in a separate repository: https://github.com/benceszasz/CareLinkJavaClient

For anyone who, like me, never really became fond of Java, here is a Python port of the Java client by @benceszasz : https://github.com/ondrej1024/carelink-python-client

It has the same functionality as the original library. Next step is to do the porting to MicroPython in order to be able to run it on constrained devices like the M5 Core2 (which would make a nice bedside display). But first tests show that this might not be trivial because lot's of things will have to be implemented manually (e.g. redirects, url parsing, ecc)

AshwinKhemlani commented 3 years ago

Hi, I am new this Nightscout system. Am I able to use nightscout to view my medtronic cgm data on my apple watch. I have a Medtronic 770G. Help would be very nice.

ondrej1024 commented 3 years ago

I started playing with the M5 Core2 device and implemented a Minimed Monitor, using the Carelink client to pull the data from the Carelink Cloud. The client is not yet integrated into the Core2 and runs on a proxy device for now. Here are pictures of the 2 screens which are implemented, with real time data. I will probably also add another screen with the history graph. Lots of details are still to be worked out.

minimed_mon_1

minimed_mon_2

AlejandroPaula commented 3 years ago

Excellent!! And have you gotten alarms to sound? I would like to implement it, have you finished the repo?

ondrej1024 commented 3 years ago

No alarms are handled yet. But it's on my list. The Core2 can beep and play audio files. So it's possible. I will set up a repository with the code, so anyone who is interested can contribute.

ondrej1024 commented 3 years ago

If anyone is interested in contributing, I created a repository for the M5Stack Minimed Monitor for the M5 Core2 device.

AlejandroPaula commented 3 years ago

Thanks for your contribution👏👏 .. can it be implemented with M5STACK ESP32?

ondrej1024 commented 3 years ago

Thanks for your contribution clapclap .. can it be implemented with M5STACK ESP32?

I'm not sure I understand what you are asking. As far as I know all M5Stack devices are based on the ESP32. And so is the M5 Core2 I am using for the Minimed Monitor.

AlejandroPaula commented 3 years ago

Hi, I am trying to implement in v Code and I am stuck, could you do a little tutorial on how to implement it ?. If you do not mind

ondrej1024 commented 3 years ago

Hi, I am trying to implement in v Code and I am stuck, could you do a little tutorial on how to implement it ?. If you do not mind

I suggest you open an issue in the repository for the M5Stack Minimed Monitor. So we don't hijack this thread here not related to minimed-connect-to-nightscout.

lhmcgann commented 3 years ago

Hello, I am new to this and trying to get minimed-connect-to-nightscout working. I am in the US using a 770G. I set up a MongoDB database, forked cgm-remote-monitor, and deployed it through Heroku (although I removed 'mmconnect' from the ENABLE env variable). I am trying to run the minimed-connect-to-nightscout directly using Node.js and npm start. I have all the CareLink login and nightscout hostname environment variables set and additionally set MMCONNECT_COUNTRYCODE=us per earlier discussions here to get past the 400 error. Now, however, I am now hitting the "Stale CareLink data: 27181314.92 minutes old" error. I saw some discussion about this here as well but couldn't determine the resolution. Any help would be much appreciated!

mattster98 commented 3 years ago

This is still unresolved, correct? I've been using the modified xdrip+ to pull from carelink and push to nightscout but it has gotten inconsistent lately. The data is on carelink but there are gaps on nightscout. Hoping for a more reliable solution.

stijnbrouwers commented 3 years ago

@mattster98

This is still unresolved, correct? I've been using the modified xdrip+ to pull from carelink and push to nightscout but it has gotten inconsistent lately. The data is on carelink but there are gaps on nightscout. Hoping for a more reliable solution.

I got it working on my PR #24. You can check that out to see if it works for you. (I tested it for My son's 780G)

ghubc commented 2 years ago

Hello, is there a way to export 780g data (sensor readings) without connecting to the Minimed server? So, is there an offline export way? I do not want for Minimed to have my sons medical informations.

We just want to get a pdf with his pump usage for last 14 days so we can get it to his doctor to discuss a diabetes treatment. Like it was possible to transfer from Medtronic 530g.

We are in EU and have Career Blue adapter USB, which came with a pump.

Thank you! :) Greetings

ondrej1024 commented 2 years ago

Hi @ghubc, Medtronic provides all the Cloud infrastructure to upload the patient data from the pump. So the doctor with a Carelink account can access it if you grant him/her permission.

The Minimed 780G uploads the data via the "Minimed mobile" app on the patients Smartphone. I think you can also use a PC with the Career Blue adapter and the official Carelink uploader software. I am not aware of any tool provided by Medtronic to process the pump data offline without uploading it to the Cloud.

That is something the Open Source community needs to implement, I guess. In theory it would be possible to pull the data directly from the pump via the BLE communication. Unfortunately the communications protocol is not published by Medtronic and there don't seem to be any efforts ongoing to reverse engineer it.

The radio communication of the older pump 670G had been reverse engineered by @pazaan and pubished here. And this opened up the possibility to include support for the pump in Open Source projects like Nightscout and Tidepool. So until this step is accomplished, I guess there is no alternative to using the official Carelink Cloud.

paul1956 commented 2 years ago

I think the 770G/780G is even more difficult because it used a modified pairing protocol to prevent hacking with the goal of eventually using phone as a remote, and giver the recent recall of 530g remote a lot of effort is going into security. Once the data is on CareLink Cloud, there are several Open-Source projects to allow downloading the raw data which is updated every 5 minutes, at every alert and every 24 hours. Medtronic on 1/7/2022 just increased the server security requiring stronger passwords. I am not sure what your concern is specifically with the data stored on a cloud there does not seem to be any way to tie it to a person, you create the user access ids and passwords and what little personal information they have, and no one verifies it.

m4esen commented 2 years ago

Very nice, thx!!! Just tested in with a 780G pump in Belgium and it's working fine with xdrip. I'm very happy!!!

Hi, may you show me what your URL looked like to sucessfully connect to carelink.minimed.eu? I do not get it to work.

Trenar commented 2 years ago

It worked for me for some weeks, but since 3-4 days the carelink.minimed.eu server is not available anymore, isn't it? At least I get an access error:

Access Denied

You don't have permission to access "http://carelink.minimed.eu/" on this server.

Reference #18.841d1002.1647170111.834554dc

Anybody else facing this issue?

Trenar commented 2 years ago

Anybody else facing this issue?

Nevermind.. It seems like I had too many requests from my IP and got banned by medtronic... After retrieving a new IP (i.e. restarting my router) everything works again.

Obviously, my covid quarantine with me and all my devices being at home for 2 weeks was too much for the server (I'm self-hosting) :D

seanch80 commented 2 years ago

4. POST to https://clcloud.minimed.com/connect/v2/display/message

Has anyone found a way to only get data for the last hour, or last half-hour, in order to limit the amount of data polled? ..and is there a limit on the polling frequency using the ble data endpoint?

I want to poll the recent data every 5 minute or so. Would that work or does that get me blocked by Medtronic?

ondrej1024 commented 2 years ago

The carelink_client_proxy.py (part of my repo carelink-python-client ) downloads the available Carelink data, including last 24 hour history, every 5 mins. I have been doing this for months and it's still running fine. So I don't think there is a rate limit, at least not in the "once every 5 min" range.

I have also been wondering if there is an API endpoint which provides only the current data (as this is what I need), but I don't think there is.

seanch80 commented 2 years ago

I have been doing this for months and it's still running fine.

Thank you for the information. I appreciate you sharing your experience with this. I will go ahead with the 5 minute polling then.

I was guessing perhaps the BLE endpoint (clcloud.minimed.com/connect/v2/display/message or equivalent) could have some optional time range parameter. In the old days I would be able to spy on the network traffic of the official Carelink Connect, but now with HTTPS, apps and smartphones I don't know how to do it.

benceszasz commented 2 years ago

@seanch80 no, there is no such option or parameter, the /connect/v2/display/message always returns the data of the last 24 hours.

seanch80 commented 2 years ago

/connect/v2/display/message always returns the data of the last 24 hours.

@benceszasz Thanks for your answer. Of course I believe you, 100%. I am not trying to be ironic, rude or sarcastic here, however it makes me curious of how you know for sure. Did you or anyone else obtain API documentation or capture the network traffic of the Carelink Connect app?

paul1956 commented 2 years ago

@seanch80 You always get 24 hours of data on this API; I have not found another API. You can look at https://github.com/paul1956/CareLink it will show you all the available data and how to get it. It currently polls every minute but only updates when something has changed by looking at lastMedicalDeviceDataUpdateServerTime. The phone app seems to be almost instant sometimes, so there may be other API's.

seanch80 commented 2 years ago

You can look at https://github.com/paul1956/CareLink it will show you all the available data and how to get it

@paul1956 Wow, that looks very nice. Thanks for the information!

paul1956 commented 2 years ago

@seanch80 thanks, still working on updating display as I continually find new information based on what pump is doing so you may get a popup sometimes. it also saves all your data in CareLinkLastDownload(en-US).json every time it updates with you language and country encoded in the name.

amspoke commented 1 year ago

HI,

I am not extremely experienced publishing things in GitHub but I have uploaded here my small personal project, to sync carelink connect with your nightscout site. It's a SAM app you can deploy using SAM from AWS. The project should be covered with the AWS free tier. So you will need an AWS account to deploy the sync app. Just give a try and give me feedback please. https://github.com/amspoke/java-carelink-lambda

ondrej1024 commented 1 year ago

As of today (after months trouble free of exercise) my carelink-python-client does not connect to the Carelink server anymore. I get the following error message:

Client login error! Response code: 504 Error message: 'sessionID'

Anyone here noticing similar issues ? Medtronic might have done some changes :worried:

mattster98 commented 1 year ago

Yes - same here. Started a couple hours ago it seems.

Getting an error trying carelink on the web, so I'm hoping it's temporary.

image
bewest commented 1 year ago

FWIW, I am looking for help moving the logic for plugins like this to https://github.com/nightscout/nightscout-connect. The benefit of nightscout-connect is that features added there will not require changes in Nightscout core.

lmbigdata commented 1 year ago

I'm having the same issue but I can access to carelink.minimed.eu

when it's not working usually this site it's not working also