nightscout / cgm-remote-monitor

nightscout web monitor
GNU Affero General Public License v3.0
2.4k stars 71.69k forks source link

Medtronic 770G/780G combined with Guardian Link 3 compatibility #6568

Closed mohammed-nep-group closed 3 years ago

mohammed-nep-group commented 3 years ago

If you need support for Nightscout, PLEASE DO NOT FILE A TICKET HERE For support, please post a question to the "CGM in The Cloud" group in Facebook (https://www.facebook.com/groups/cgminthecloud) or visit the WeAreNotWaiting Discord at https://discord.gg/zg7CvCQ

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Yeah I can't get data from my current system/from carelink to Nightscout

Describe the solution you'd like A clear and concise description of what you want to happen.

I would like to pull data from carelink which stores the 780G and Guardian Link 3 Or maybe even pull it from the pump since this is an Bluetooth enabled one. Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered. Pull data from the insulin pump itself since this is Bluetooth enabled, don't know if this is already possible. Additional context Add any other context or screenshots about the feature request here.

iwanvandaele commented 3 years ago

Same here

mohammed-nep-group commented 3 years ago

I think this is related to my ticket : https://github.com/nightscout/cgm-remote-monitor/issues/6520

Maybe also related : https://github.com/NightscoutFoundation/xDrip/issues/1473

michaelholderman commented 3 years ago

Same here. Is this something a developer can do, or is it a limitation posed by Medtronic? (I believe you could setup Nightscout with their stand alone Guardian Connect system, which uses the same process as the 770/780 as far as data upload to Carelink.)

michaelholderman commented 3 years ago

I would love to help figure out how to get Nightscout to work with the 770/780g and Carelink. I have been using Nightscout for years with the 670g and also with the Guardian Connect system and now have a couple of 770g systems that I can use to test. Just not sure where to begin. Is there someone that has worked in this in the past that I could reach out to?

benceszasz commented 3 years ago

@michaelholderman if you have 770Gs, which you can use to test, then please check the CareLink device family of the 770G in CareLink, it is returned by the /patient/monitor/data REST API. You can use for example the network traffic monitor of a browser: /patient/monitor/data REST API is called, when you click on the Connect link in CareLink. The DeviceFamily can be found in the response of the https://carelink.minimed.com/patient/monitor/data request. For example with Chrome you can use the Network Traffic monitor in the Developer Tools (More Tools > Developer Tools).

mohammed-nep-group commented 3 years ago

@benceszasz tried to use the api endpoint but resulted in :

{ "error": { "type": "InvalidToken", "group": "AUTH" } }

I don't know how the Authorization needs to be passed to Carelink. I already tried Basic auth , didn't work.

mohammed-nep-group commented 3 years ago

I have the 780G and would love to help, let me know where I can help, If I do a GET to : https://carelink.minimed.com/patient/monitor/data With a Bearer token which I took in a request from me to Carelink. I get the following message: Not Found

I think I need to pass a parameter but don't know what. In order to define who am I looking data for.

mohammed-nep-group commented 3 years ago

@benceszasz however I found the following endpoint does return a GET result with adding the Bearer token : https://carelink.minimed.eu/patient/dataUpload/recentUploads?numUploads=40

In this case 40 times in JSON. { "date" : 1607128862, "mobileUploaded" : true, "device" : "MiniMed 780G", "serialNumber" : "_serial_number_removed_it_self" }, { "date" : 1607042518, "mobileUploaded" : true, "device" : "MiniMed 780G", "serialNumber" : "_serial_number_removed_it_self" }

mohammed-nep-group commented 3 years ago

@michaelholderman if you have 770Gs, which you can use to test, then please check the CareLink device family of the 770G in CareLink, it is returned by the /patient/monitor/data REST API. You can use for example the network traffic monitor of a browser: /patient/monitor/data REST API is called, when you click on the Connect link in CareLink. The DeviceFamily can be found in the response of the https://carelink.minimed.com/patient/monitor/data request. For example with Chrome you can use the Network Traffic monitor in the Developer Tools (More Tools > Developer Tools).

I found in the endpoints you specified the following, I think that's what you're looking for :

{ "deviceFamily" : "BLE_X" }

ondrej1024 commented 3 years ago

Hi all, I would like to join the development efforts to implement an open source solution for reading the sensor and pump data from the 770G/780G. Before going into details I would like to understand how the official Medtronic system works and where we can hook up our community system. I made a simple drawing which shows how I think the system is working right now:

minimed 780g comms

So my first question is: Is my understanding of this correct?

If the answer to that question is yes, then I see 2 possibilities to get the data out of the system. The first is the Bluetooth connection (1) and the second the Carelink REST API (2).

Is there any documentation available for either of these 2? This is important to know because if there is none it will have to be a reverse engineering effort which is probably quite a long shot. So any information you might have would be very welcome.

mohammed-nep-group commented 3 years ago

@ondrej1024 Hi man, Nice that you're also joining the effort to fix this.

I think your image is indeed correct, as far as I know couldn't find any documentation. But I am going to call or mail with Medtronic minimed.

mohammed-nep-group commented 3 years ago

@ondrej1024 called with Medtronic but they told me they can't provide me with anything.

So I think the reverse engineering is the only way. Also since Medtronic is strict on this.

ondrej1024 commented 3 years ago

That was to be expected. These companies don't collaborate with the open source community without specific agreements. :disappointed:

I have contacted Tidepool.org to find out if they are working on support for the 770G/780G for their uploader software which supports all the previous Medtronic pumps. They have links to Medtronic. Hope to get some answer soon.

If there is no documentation whatsoever it will get very tricky. As for the pumps Bluetooth based radio communication, I guess we need dumps of the Bluetooth packets exchanged between the pump and the Minimed Mobile app. But that communication is probably encrypted. So if anyone has experience with that ...

To give you an idea of what we are talking about, here is a document describing the packet structure of the 640G/670G pumps radio based communication (which has also been reverse engineered).

benceszasz commented 3 years ago

@ondrej1024 reverse engineering the complete Bluetooth communication and pairing process between the 7xxG pumps and the Minimed Mobile application seems very complicated for me. Wouldn't it be "easier" to either try to "patch" the official MiniMed Mobile / CareLink Connect application like the patched Dexcom app / LibreLink app / Eversense app or try to reverse engineer the REST API communication with CareLink servers?

ondrej1024 commented 3 years ago

@benceszasz

Wouldn't it be "easier" to either try to "patch" the official MiniMed Mobile / CareLink Connect application like the patched Dexcom app / LibreLink app / Eversense app

What exactly do yo mean by patching the official apps? These are proprietary. We don't have their source code.

or try to reverse engineer the REST API communication with CareLink servers?

Yes, this is probably easier, although the HTTP traffic is surely encrypted and at the moment I have no idea how to work around that.

But in the end I would prefer being able to pull the data directly from the pump via the Bluetooth link to be completely independent from the Minimed mobile app and the Carelink cloud.

benceszasz commented 3 years ago

@ondrej1024 by patching I mean to modify the offical application in a way to extract data from it just like in the case of the other CGM applications, which I have mentioned: read the internal database of the application or inject a custom code (for example an event listener to receive and forward the data to another app (eg. xDrip+) or NS). Of course the source code is not available, but there is the APK.

mohammed-nep-group commented 3 years ago

Oké Guys, what are now the possibilities? Because I am still interested to fix this. @ondrej1024 @benceszasz

pazaan commented 3 years ago

The CareLink Connect app is using Firebase (and therefore Protobufs as the communication protocol), and has so far been resilient to the simple attacks (straight MITM, attempts to run it on a rooted Android device). There are further possibilities to try, but they start to get quite a bit more involved and low-level.

Longer term, I would say that we would want to RE the BLE, since reverse engineering the API still leaves data collection dependent on CareLink servers, which hasn't been as reliable up to this point that we'd like it to be 😉. I would expect that the underlying app-level protocol is the same (or very similar) to the 600-series protocol. It's just the higher level BLE layer that needs to be unwrapped. I don't have any background for sniffing BLE. Also, I don't have a 780G, so I don't even have a place to start.

ondrej1024 commented 3 years ago

Hi Lennart, don't you think that Tidepool and Medtronic might reach some agreement as to documentation of the BLE based protocol ? Or is Tidepool just hoping that sooner or later someone will reverse engineer it so they can in also integrate it into the TP uploader?

pazaan commented 3 years ago

I'm not sure. It's possible, but I certainly wouldn't rely on it if you would like data in Nightscout sooner rather than later.

markxoe commented 3 years ago

I am not sure if this helps, but i just saw this: https://github.com/nightscout/minimed-connect-to-nightscout

mohammed-nep-group commented 3 years ago

I am not sure if this helps, but i just saw this: https://github.com/nightscout/minimed-connect-to-nightscout

This is only working up to the 670g as I understand. The 770/780G series are a little bit different.

I already tried to let it work but didn't succeed with the data extractor you mentioned in the nightscout project.

markxoe commented 3 years ago

The 670G doesn't have a iphone app, I think it is for the 770G, maybe someone can try? I only have the 670G 😬

ondrej1024 commented 3 years ago

@mohammed-nep-group , if you have a 770G or 780G you could start by dumping the BLE packets exchanged between the pump and the Minimed Mobile app on the Smartphone. If your phone is Android you could start here for instructions about packet sniffing.

mohammed-nep-group commented 3 years ago

I do have the 780G and are already on it 2/3 weeks. So I do know how it works. I only use an Iphone. Will be looking for a BLE sniffer for that. @ondrej1024

mohammed-nep-group commented 3 years ago

I see there are possibilities on Iphone : https://www.bluetooth.com/blog/a-new-way-to-debug-iosbluetooth-applications/ As soon as I got Bluetooth sniffing logs, I will let it know here.

mohammed-nep-group commented 3 years ago
Schermafbeelding 2020-12-15 om 19 48 24

You can see here a part of the Bleutooth Logging. I filtered only on the pump data.

mohammed-nep-group commented 3 years ago

pump data .txt

Also here is a txt dump of some time logging the Bluetooth device. @ondrej1024

ondrej1024 commented 3 years ago

Very interesting. No we have to make sense of it :smirk:

Assuming that the basic communication principles of this pump are the same as the 670G i would expect to see a pump status message every 5mins. The 670G only answers to requests but with BLE I guess the 780G could send it's status message whenever it has new data. However I am not really familiar with the BLE protocol.

In the log (txt file) I see basically 5 blocks of data exchange, based on the time stamps. While the first 2 are different, the last 3 seem to be very much the same: 13 messages repeated every minute with the same sequence, 19:48:38, 19:50:38 and 19:51:38. The message block from 19:49:38 is missing, though :thinking: Remains to be seen what these blocks are.

The first 2 message blocks could be the pump status report. The first block lasts 5s from 19:47:25 to 19:47:30. The second blocks starts 5s later and lasts 7s from 19:47:35 to 19:47:42. We need to check if we can recognize the pattern of the 670G pump status message in there.

mohammed-nep-group commented 3 years ago

I do agree, because if we see the same pattern of the 670G It would be a little bit easier.

ondrej1024 commented 3 years ago

Could you provide another log for a duration of about 10min?

mohammed-nep-group commented 3 years ago

Could you provide another log for a duration of about 10min?

Yes will do it right now. @ondrej1024

mattmholliday commented 3 years ago

I'm keen to help where I can. I have a 770G but my android phone is not compatible with the minimed app

ondrej1024 commented 3 years ago

I'm keen to help where I can. I have a 770G but my android phone is not compatible with the minimed app

First step: get yourself a compatible phone :wink: But seriously, I think the whole point of the 770G with respect to its predecessors is the Bluetooth connectivity and the app.

mattmholliday commented 3 years ago

@ondrej1024 I might skip that step ;)

I've got the bluetooth uploader setup on my PC

ondrej1024 commented 3 years ago

I've got the bluetooth uploader setup on my PC

What's that? A software provided by Medtronic? Similar to their java uploader for the 670G?

mattmholliday commented 3 years ago

I've got the bluetooth uploader setup on my PC

What's that? A software provided by Medtronic? Similar to their java uploader for the 670G?

Correct. You hold down the bottom arrow on the pump to get the paring code before you upload.

ondrej1024 commented 3 years ago

That could be interesting to see how the history download from he pump works. You can do the sniffing of the BLE packets directly on the PC for that.

mattmholliday commented 3 years ago

Happy to give it a try. I'm currently digging through the uploader.jar file to see what they have added for the ble connection. Can you recommend a tool to use for capturing the BLE packets?

ondrej1024 commented 3 years ago

Unfortunately I'm not familiar with any of the Bluetooth stuff. Still have to learn this myself.

mattmholliday commented 3 years ago

Fair enough. I'm trying to learn about BLE and GATT myself.

ondrej1024 commented 3 years ago

Could you provide another log for a duration of about 10min?

Yes will do it right now. @ondrej1024

@mohammed-nep-group are you still there? Did you manage to record a 10min BLE packets log from your pump?

ondrej1024 commented 3 years ago

I wonder why everyone here has suddenly fallen silent? Is there some activity regarding a Minimed 700 series comms driver going on elsewhere? Or has interest in this subject just fainted? Just curious :thinking:

sulkaharo commented 3 years ago

I suspect this is simply that 1) this is a complicated problem to solve and 2) AFAIK none of the previous frequent contributors to Nightscout have access to the pump, so I'm not sure how many people even could help with this even if they wanted.

ondrej1024 commented 3 years ago

or maybe 3) the official Medtronic apps (uploader and monitor) work fairly well and already provide everything most people need.

gerardoabal commented 3 years ago

@ondrej1024 very interesting thread. My son has T1D since 2 years old. We have been using Nightscout uploader with CNL with 640g model during 5 years. Now we're are excited moving to 780g pump, and the possibility to have an uploader with just an 4G/smartwatch it's awesome. But according to this thread, there is a lot of work to do. Let me know if I can help, and if the conversation has moved to any other site.

ClifClimber commented 3 years ago

Hi all ,

I am new to the 780G pump system. Until now I had used nigtgscout + androidaps + roche insigth, so I really miss all the information that nigshcout gave me (watch, reports still so on).

I want to collaborate as much as possible and I have computer skills (systems, not development).

What can I do ?

Thank you for all your work ! !

ondrej1024 commented 3 years ago

IMHO we should start by looking at the BLE packets exchanged between the pump and the Minimed Mobile app on the phone and try to see if we can recognize any of the data packets from the 600 series protocol. There is one short log here which was posted earlier in this thread.

Another approach could be to disassemble the Carelink Java uploader and see how it handles the 700 series pumps communication.

ClifClimber commented 3 years ago

Hi ondrej1024 :

do you knwo how can sniffer the BLE packets between the pump and the android sysmten?. Where I tried to Enable Bluetooth HCI snoop log the Minimd Mobile App stops working with the message : "This app does not work with developer options active"

Regards

ondrej1024 commented 3 years ago

the Minimd Mobile App stops working with the message : "This app does not work with developer options active"

Looks like Medtronic wants to make this difficult. But we knew this, didn't we? :disappointed:
@mohammed-nep-group got the BLE sniffing working on an iPhone (see above) but seems to have disappeared. I don't know if anyone else tried on Android yet. Unfortunately I have no experience with that.