Bolus detected as Carb Correction (sometimes)

[burnedikt]

This is a follow-up issue to https://github.com/burnedikt/diasend-nightscout-bridge/issues/15. Seems like the issue still exists, albeit it happens more rarely.

As seen in the image, the second bolus was incorrectly detected as a carb correction:

The underlying data from diasend looks fine on first sight:

[
  {
    "type": "insulin_bolus",
    "created_at": "2022-12-01T11:16:08",
    "unit": "U",
    "total_value": 0.2,
    "spike_value": 0.2,
    "suggested": 0.2,
    "suggestion_overridden": "no",
    "suggestion_based_on_bg": "no",
    "suggestion_based_on_carb": "yes",
    "programmed_meal": 0.2,
    "flags": [
      {
        "flag": 1035,
        "description": "Bolus type ezcarb"
      }
    ]
  },
  {
    "type": "carb",
    "created_at": "2022-12-01T11:16:16",
    "value": "5",
    "unit": "g",
    "flags": []
  },
  {
    "type": "glucose",
    "created_at": "2022-12-01T11:18:50",
    "value": 259,
    "unit": "mg\/dl",
    "flags": [
      {
        "flag": 123,
        "description": "Continous reading"
      }
    ]
  },
  {
    "type": "insulin_bolus",
    "created_at": "2022-12-01T11:20:50",
    "unit": "U",
    "total_value": 0.4,
    "spike_value": 0.4,
    "suggested": 0.4,
    "suggestion_overridden": "no",
    "suggestion_based_on_bg": "no",
    "suggestion_based_on_carb": "yes",
    "programmed_meal": 0.4,
    "flags": [
      {
        "flag": 1035,
        "description": "Bolus type ezcarb"
      }
    ]
  },
  {
    "type": "carb",
    "created_at": "2022-12-01T11:20:57",
    "value": "10",
    "unit": "g",
    "flags": []
  },
  {
    "type": "glucose",
    "created_at": "2022-12-01T11:23:50",
    "value": 264,
    "unit": "mg\/dl",
    "flags": [
      {
        "flag": 123,
        "description": "Continous reading"
      }
    ]
  }
]

[burnedikt]

These are the logs of processing:

[
  {
    "timestamp": "2022-12-01T10:15:23.351450000Z",
    "message": "Number of glucose records since 12 minutes ago (2022-12-01T10:03:51.000Z - 2022-12-01T10:15:22.826Z):  2"
  },
  {
    "timestamp": "2022-12-01T10:15:23.351549000Z",
    "message": "Sending 2 entries to nightscout"
  },
  {
    "timestamp": "2022-12-01T10:15:23.504702000Z",
    "message": "Next run (Entries) will be in in 2 minutes..."
  },
  {
    "timestamp": "2022-12-01T10:17:01.656999000Z",
    "message": "Sending 1 treatments to nightscout"
  },
  {
    "timestamp": "2022-12-01T10:17:01.657426000Z",
    "message": "Updating basal profile based on 0 records"
  },
  {
    "timestamp": "2022-12-01T10:17:01.838456000Z",
    "message": "Scheduling 0 records for processing in next run"
  },
  {
    "timestamp": "2022-12-01T10:17:01.838853000Z",
    "message": "Next run (Treatments) will be in in 2 minutes..."
  },
  {
    "timestamp": "2022-12-01T10:17:24.122621000Z",
    "message": "Number of glucose records since 4 minutes ago (2022-12-01T10:13:51.000Z - 2022-12-01T10:17:23.505Z):  0"
  },
  {
    "timestamp": "2022-12-01T10:17:24.122722000Z",
    "message": "Sending 0 entries to nightscout"
  },
  {
    "timestamp": "2022-12-01T10:17:24.122865000Z",
    "message": "Next run (Entries) will be in in 2 minutes..."
  },
  {
    "timestamp": "2022-12-01T10:19:02.595088000Z",
    "message": "Sending 0 treatments to nightscout"
  },
  {
    "timestamp": "2022-12-01T10:19:02.595386000Z",
    "message": "Updating basal profile based on 0 records"
  },
  {
    "timestamp": "2022-12-01T10:19:02.742060000Z",
    "message": "Scheduling 0 records for processing in next run"
  },
  {
    "timestamp": "2022-12-01T10:19:24.675450000Z",
    "message": "Number of glucose records since 6 minutes ago (2022-12-01T10:13:51.000Z - 2022-12-01T10:19:24.125Z):  1"
  },
  {
    "timestamp": "2022-12-01T10:19:24.675556000Z",
    "message": "Sending 1 entries to nightscout"
  },
  {
    "timestamp": "2022-12-01T10:19:24.831616000Z",
    "message": "Next run (Entries) will be in in 2 minutes..."
  },
  {
    "timestamp": "2022-12-01T10:21:03.456858000Z",
    "message": "Sending 0 treatments to nightscout"
  },
  {
    "timestamp": "2022-12-01T10:21:03.457251000Z",
    "message": "Updating basal profile based on 0 records"
  },
  {
    "timestamp": "2022-12-01T10:21:03.606032000Z",
    "message": "Scheduling 0 records for processing in next run"
  },
  {
    "timestamp": "2022-12-01T10:21:03.606437000Z",
    "message": "Next run (Treatments) will be in in 2 minutes..."
  },
  {
    "timestamp": "2022-12-01T10:21:25.552543000Z",
    "message": "Number of glucose records since 3 minutes ago (2022-12-01T10:18:51.000Z - 2022-12-01T10:21:24.833Z):  0"
  },
  {
    "timestamp": "2022-12-01T10:21:25.552695000Z",
    "message": "Sending 0 entries to nightscout"
  },
  {
    "timestamp": "2022-12-01T10:21:25.552748000Z",
    "message": "Next run (Entries) will be in in 2 minutes..."
  },
  {
    "timestamp": "2022-12-01T10:23:04.326820000Z",
    "message": "Sending 0 treatments to nightscout"
  },
  {
    "timestamp": "2022-12-01T10:23:04.327231000Z",
    "message": "Updating basal profile based on 0 records"
  },
  {
    "timestamp": "2022-12-01T10:23:04.471602000Z",
    "message": "Scheduling 1 records for processing in next run"
  },
  {
    "timestamp": "2022-12-01T10:23:04.471927000Z",
    "message": "Next run (Treatments) will be in in 2 minutes..."
  },
  {
    "timestamp": "2022-12-01T10:23:26.135848000Z",
    "message": "Number of glucose records since 5 minutes ago (2022-12-01T10:18:51.000Z - 2022-12-01T10:23:25.553Z):  0"
  },
  {
    "timestamp": "2022-12-01T10:23:26.135920000Z",
    "message": "Sending 0 entries to nightscout"
  },
  {
    "timestamp": "2022-12-01T10:23:26.136032000Z",
    "message": "Next run (Entries) will be in in 2 minutes..."
  },
  {
    "timestamp": "2022-12-01T10:25:05.181842000Z",
    "message": "Sending 1 treatments to nightscout"
  },
  {
    "timestamp": "2022-12-01T10:25:05.182271000Z",
    "message": "Updating basal profile based on 0 records"
  },
  {
    "timestamp": "2022-12-01T10:25:05.333115000Z",
    "message": "Scheduling 0 records for processing in next run"
  },
  {
    "timestamp": "2022-12-01T10:25:05.333566000Z",
    "message": "Next run (Treatments) will be in in 2 minutes..."
  },
  {
    "timestamp": "2022-12-01T10:25:26.737578000Z",
    "message": "Number of glucose records since 7 minutes ago (2022-12-01T10:18:51.000Z - 2022-12-01T10:25:26.137Z):  1"
  },
  {
    "timestamp": "2022-12-01T10:25:26.737686000Z",
    "message": "Sending 1 entries to nightscout"
  },
  {
    "timestamp": "2022-12-01T10:25:26.889155000Z",
    "message": "Next run (Entries) will be in in 2 minutes..."
  }
]

and the database shows this:

[
  {
    "_id": { "$oid": "63888101287828157feeb816" },
    "eventType": "Carb Correction",
    "carbs": { "$numberInt": "10" },
    "app": "diasend",
    "date": { "$numberDouble": "1.6698900570000E+12" },
    "created_at": "2022-12-01T10:25:05.181Z",
    "utcOffset": { "$numberInt": "60" }
  },
  {
    "_id": { "$oid": "63887f1d287828157fe87c9b" },
    "eventType": "Meal Bolus",
    "insulin": { "$numberDouble": "0.2" },
    "carbs": { "$numberInt": "5" },
    "app": "diasend",
    "date": { "$numberDouble": "1.6698897680000E+12" },
    "created_at": "2022-12-01T10:17:01.658Z",
    "utcOffset": { "$numberInt": "60" }
  }
]

---

The interesting part here is that the bolus was given at 2022-12-01T10:20:57Z but it was only entered into the database at 2022-12-01T10:25:05.181Z which means 5 minutes later. Likely because the diasend API did not return the result earlier (and CamAPS Fx only uploads data to diasend every 5 minutes). Then, the only question is why the bolus is not related to the carbs. Checking the logs, we can see that at 2022-12-01T10:23:04.471602000Z, the bridge already found a record that couldn't be handled (i.e. a bolus withour matching carbs) which was scheduled to be revisited on the next run. This unprocessed record would either have either been the carbs record (which we always postpone to be checked again in the next run) or the bolus for which no matching carbs were available (yet). Since on the next run, the carb record was entered and we afterwards had no more unprocessed records, it apparently was the carb record. And since the bridge only fetches records AFTER the last known record of the last iteration, it would never find the bolus as this is technically returned before the carbs record.

So my assumption right now is that:

• the bridge fetches records from the diasend API for the last 2 minutes. Diasend only returns the carb record, the bolus is not yet contained within the API response.
• Bridge remembers the carb record for the next run
• in the next run, 2 minutes later, bridge fetches the records from the API, limiting the results to events happening after the already known carb record and the current time
• the bolus record was by now added to the diasend API but it's timestamp is prior to the carb record and will thus never be picked up

---

To validate that this assumption is true, we'd need to monitor the diasend API whether it really inserts records at an earlier time later on. To check this, we'd need to poll the API, detect a carb record and then start monitoring the API for the time +/- 1 minute around the carb record to see whether a bolus record is added later on. If that's the case, we'd also need to adapt the algorithm so that it will search for events prior to the last known record as well (in case it's an unprocessed record). The difficulty here will be to prevent handling records twice again ...

[burnedikt]

To validate that this assumption is true, we'd need to monitor the diasend API whether it really inserts records at an earlier time later on. To check this, we'd need to poll the API, detect a carb record and then start monitoring the API for the time +/- 1 minute around the carb record to see whether a bolus record is added later on.

I recorded what the diasend API returns for the last 24 hours and the findings are as I suspected. Basically, carb records appear to be always available via the API first, the corresponding bolus records (if any) will only be available via the API 5 to 10 minutes later - and what's important: it's added within the API response before the carb record.

Example: Retrieve the patient records from diasend for the timespan 2022-12-03T16:05:00 - 2022-12-03T16:15:00 at 2022-12-03T16:13:40 and you get:

[{
    "type": "carb",
    "created_at": "2022-12-03T16:10:52",
    "value": "16",
    "unit": "g",
    "flags": [],
}]

Now you query the API again for the same timespan but at a later time 2022-12-03T16:17:40 (i.e. 4 minutes later) and suddenly you get:

[{
    "type": "insulin_bolus",
    "created_at": "2022-12-03T16:10:44",
    "unit": "U",
    "total_value": 0.6,
    "spike_value": 0.6,
    "suggested": 0.6,
    "suggestion_overridden": "no",
    "suggestion_based_on_bg": "no",
    "suggestion_based_on_carb": "yes",
    "programmed_meal": 0.6,
    "flags": [{ "flag": 1035, "description": "Bolus type ezcarb" }]
  },
  {
    "type": "carb",
    "created_at": "2022-12-03T16:10:52",
    "value": "16",
    "unit": "g",
    "flags": []
}]

So the insulin bolus record has been added to the api response prior to the carb record, 4 minutes later. At that time, our algorithm wouldn't check the timespan again for new records.

Based on this, the current implementation is doomed to fail as it only looks ahead, i.e. for future added boli that belong to a carb record but are returned within the API after the carb record, not before.

---

General problem statement is: Records are added to the diasend API to timespans that have already been handled by the bridge and are therefore never looked at again. These "late-added" records are lost and will not be processed.

This issue mainly affects meal boli (and therefore insulin_bolus and carb records) as for those, we need to match two different record types. For other record types, the issue is less prevalent or non-existent at all (e.g. a late-added insulin_basal or glucose record). Carb records on their own cannot be used to correctly identify whether they are carb corrections or belong to a meal boli as we need the corresponding insulin_bolus record for that (which might not yet be within the API response).

Probably the easiest way to fix this is to look at a larger time interval going back at least 10 minutes in each iteration to catch any late-added records and process them correctly. Right now, we only look at any events that happened between the last run of the synchronization loop and the current time. In the future, we could look back 15 minutes from the current time. We'd then need to make sure to not handle any of the records twice (as we would fetch them multiple times with the overlapping time intervals). This is probably only possible by keeping a list of the records ourselves ("shadow records") internally.

[burnedikt]

Today, I've noticed an additional potential issue:

When a bolus is manually given via the pump, that bolus likely does not get imported to nightscout. It shows up on CamAPS Fx and diasend, though. However, it is only imported at a later point in time (presumably when CamAPS polls the pump the next time).

This polling cycle between the app and the pump is probably also why carbs show up earlier on diasend than the corresponding bolus: The app directly sends carbs to diasend but waits for the pump to confirm that the bolus was in fact given, before reporting it to diasend (which is sensible behaviour, just didn't think of it yet)...

[burnedikt]

The solution for the issue I have in mind right now would be to delay loading and processing of (carb and bolus) records by 10 to 15 minutes to make sure we're not missing out on records added late to diasend.

Even then, though, there can be cases where a bolus will only be added an hour or even more later, e.g. because no connection to the pump for some time. By that time, our polling loop will already assume that timespan is handled and all records are processed, which is wrong.

So not only do we need to delay the synchronization of bolus and carb records from diasend to Nightscout but we'll also every few hours need to perform a "correction sync" where we compare to what's on diasend to what's on Nightscout already and update existing / add missing treatments (e.g. when four hours have passed, revisit the same timespan again to find out anything that has been added retrospectively). This only applies to treatments imho as for glucose values, missing out on a value is not critical but missing out on a bolus or carb correction messes up all statistics.

The delayed processing of records will ensure we still have near-real-time (10 to 30 minutes delay) import of boli and carb corrections, the above mentioned correction sync should additionally ensure we're not missing out on edge cases.

[burnedikt]

Version 1.1.2 implements a naive approach of this by allowing to specify a delay for processing treatments. This can be used like:

startSynchronization({
  // set to however many milliseconds you want to delay the fetching and processing of treatments. Will not affect glucose record synchronization
  // in this case, treatments will be processed only with a 15 minutes delay, i.e. at 11:00, the bridge will synchronize records reported by diasend around 10:45
  treatmentsLoopDelayMs: 15 * 60 * 1000
})

I am not super happy with this, as it will also "slow" down the processing of e.g. temp basal change records where it does not seem necessary. But for now, I mainly want to see whether it fixes the issue at hand with boli not imported correctly so 1.1.2 focusses on eventual data consistency between diasend and nightscout more than on importing in a timely manner.

---

Edit: Updated version from 1.1.0 to 1.1.2 due to a bug not delaying the treatments but glucose loop instead

[burnedikt]

Running Version 1.1.2 with the delayed treatments loop for a couple of days showed that it indeed is less likely to incorrectly classify boli (or miss out on those in the diasend API), however, the delay in events bubbling into nightscout is weird as the glucose data is there way earlier than the bolus / carbs data now. Also, it delays the processing of all treatments needlessly, as some treatments (in particular insulin_basal records but also some bolus records) appear quite timely on the diasend api response, just some outliers don't.

So to catch the outliers, processing of all treatments is slowed down, which is problematic if you're using nightscout for near-real-time monitoring. It's less of an issue if you just use it for reporting. But personally I don't care about the reporting as diasend's reporting is good enough imo, so my focus lies on the monitoring aspect.

Thus, I am thinking about implementing an alternative approach which should be both more accurate (correctly detect bolus and carb corrections) while processing events in a timely manner (if possible). The idea is to always fetch the patient records from diasend, starting at the time of the earliest out of the latest known records of each type: This means, we fetch the patient records from diasend, process them as usual (insulin_basal become Temp Basal treatments on nightcout, glucose records become entries on nightscout, insulin_bolus and carb records become Correction Bolus, Meal Bolus or Carb Correction treatments on nightscout). For each record type where there's a resulting entry on nightscout, we'll remember the latest one (i.e. most recent bolus, basal, glucose or carb record). Thereby we get one "event stream" per type of treatment / diasend patient record type. To not miss out on any records, we'll always fetch data starting at the time of the earliest of all event streams' latest record.

An example: We fetch data from diasend for the timespan 12:00 to 13:00 we get 12 glucose records, five temporary basal changes and one carb record. The glucose and basal events will be handled and forwarded to nightscout. The carb record cannot be handled yet unless it's more than 10 minutes ago (as there could be a corresponding bolus still missing otherwise). So after processing that period, the next run (e.g. 5 minutes later), will start fetching data again at 12:00as the event stream for bolus / carb is still empty. Now we're fetching data between 12:00and 13:05. By now, the missing bolus record has been added and can be matched with the carb record. The event stream for bolus / carb records will thus both now get an entry (bolus was given e.g. at 12:57). The already processed records for glucose and basal insulin are discarded (as we know the latest records for their event streams) and only newly added glucose records are added. In this case, let's assume a new glucose record was added, no basal changes. Latest basal change was at 12:53. The next iteration 5 minutes later will now fetch data between 12:53 (which is the earliest of all the latest events in each stream) and 13:10. No new events here, so next iteration will fetch data between 12:53 and 13:15. If a new temp basal record was now added the lower time limit would advance accordingly. In essence we:

• fetch all records from diasend starting at the latest known bolus, temp basal or glucose record (we pick the earliest of these records)
• split the fetched records into the different event types (bolus + carbs, temp basal, glucose record)
• process the different event types indpendently, remembering the latest processed record
• start over

As an added benefit, it will reduce the number of API calls made to diasend per synchronization iteration to 1 instead of 2.

[Jason-Burke]

@burnedikt I am experiencing the issue described in #26, no meal boli. I have v1.1.5 running on Railway.

That said, I have observed the time difference between diasend logging "insulin_bolus" and "carb" to be consistently between 2 and 3 minutes. Since it is outside the +/- 2 minute defined window, I have assumed bolus values are not processed.

My intended approach would be to modify the constant diasendBolusCarbTimeDifferenceThresholdMilliseconds in adapter.ts, to:

// carbs should have been recorded within two minutes after / before a meal bolus const diasendBolusCarbTimeDifferenceThresholdMilliseconds = 3 * 60 * 1000; const nightscoutApp = "diasend";

would you recommend the same or a different approach?

I have deployed your cgm-remote-monitor and am uncertain to the steps to modify the constant (in adapter.ts)

My guess, and it is simply that, is that I need to fork the diasend-nightscout-bridge to my GitHub in addition to your cgm-remote-monitor, make the edit to adapter.ts. Once the edit is complete goto to railway and "Redeploy" the cgm-remote-monitor which would load the bridge from my GitHub?!

[burnedikt]

@burnedikt I am experiencing the issue described in #26, no meal boli. I have v1.1.5 running on Railway.

That said, I have observed the time difference between diasend logging "insulin_bolus" and "carb" to be consistently between 2 and 3 minutes. Since it is outside the +/- 2 minute defined window, the bolus values are not processed.

My intended approach would be to modify the constant diasendBolusCarbTimeDifferenceThresholdMilliseconds in adapter.ts, to:

// carbs should have been recorded within two minutes after / before a meal bolus const diasendBolusCarbTimeDifferenceThresholdMilliseconds = 3 * 60 * 1000; const nightscoutApp = "diasend";

would you recommend the same or a different approach?

I have deployed your cgm-remote-monitor and am uncertain to the steps to modify the constant. I have naively assumed that following the merge, I cannot edit adapter.ts? At least, I can not locate it in my fork.

My guess, and it is simply that, is that I need to fork the diasend-nightscout-bridge to my GitHub, make the edit to adapter.ts in mine, goto to railway and "Redeploy" the cgm-remote-monitor which would then, in turn, load the bridge from my GitHub?!

it's a bit more complicated than that if you're running the bridge as part of nightscout directly instead of as standalone. In standalone, you could easily modify the settings and restart. But if you're using the cgm-remote-monitor fork, that one is hard linked via package.json to load the most recent published version of the bridge and you can't really modify that version.

One thing you can try is change the package.json on your cgm-remote-monitor fork to make sure it does not use the public version of the bridge (e.g. 1.0.0) but instead use your forked and modified version directly from github. This can be done by changing the dependency to not be pulled from the public npm repository but your github repository instead like, e.g. (refer to the documentation of npm for possible options):

- "diasend-nightscout-bridge": "1.1.5",
+ "diasend-nightscout-bridge": "jason-burke/diasend-nightscout-bridge",

This will install the diasend-nightscout-bridge within your nightscout installation from your fork of the bridge's main branch. As long as the fork's public, that is.

---

However, that being said, I am very open to accepting pull requests with changes like this (increasing the threshold) so you can just stick to the main version of diasend-nightscout-bridge and still have the increased threshold.

[burnedikt]

@Jason-Burke Version 1.1.6 of the bridge increases the threshold to 3 minutes just as you suggested.

From your experience and the data you see on diasend: Do the carbs belonging to a meal bolus always come after the bolus? Asking because right now, we're always looking at the timespan +/-3 minutes around the bolus time. But from my current perspective, it looks like the carbs always come after the bolus.

[burnedikt]

@Jason-Burke Version 1.1.6 of the bridge increases the threshold to 3 minutes just as you suggested.

From your experience and the data you see on diasend: Do the carbs belonging to a meal bolus always come after the bolus? Asking because right now, we're always looking at the timespan +/-3 minutes around the bolus time. But from my current perspective, it looks like the carbs always come after the bolus.

nevermind that, checked again and it still happens that the carb record is returned within the API prior to the bolus

[MiGerth]

In my experince the carbs comes always first and sometimes the corresponding bolus some minutes later. If I enter ony carbs as a small meal or a hypo I can see them in diasend immediately.

[burnedikt]

In my experince the carbs comes always first and sometimes the corresponding bolus some minutes later. If I enter ony carbs as a small meal or a hypo I can see them in diasend immediately.

yes, as far as I have reverse engineered the CamAPS app: It sends the carbs entered into the app directly to diasend, but waits for the bolus to be confirmed by the pump before sending it to diasend. And since it only talks to the pump ever so often, this might take a few minutes.

But there's two different phenomenons at play here:

1. the carbs will always appear on diasend earlier than the bolus just like @MiGerth described, however ...
2. ... the time of the bolus, once available on diasend is often earlier than the carb. So even though the carbs appear earlier on diasend, temporally speaking, they are earlier within the api response

Both problems are not an issue for e.g. the diasend app as it just loads the data once when the app is opened and then shows what's there (e.g. only the carbs if bolus not yet on diasend). But for the diasend-nightscout-bridge, this is a bit of a challenge as we fetch the data from diasend every few minutes, send the data to nightscout and then assume we already have all the data synchronized for that last few minutes but instead, some data like the bolus gets added to the diasend api at a later point in time with a timestamp that's earlier and that's how they get missed out on.

[Jason-Burke]

@burnedikt I noticed the update (v1.1.7) and your notes. I have merged and success! I now have meal boli being captured in Nightscout. Thanks, as always, for the consideration and update.

[Jason-Burke]

@Jason-Burke Version 1.1.6 of the bridge increases the threshold to 3 minutes just as you suggested.

From your experience and the data you see on diasend: Do the carbs belonging to a meal bolus always come after the bolus? Asking because right now, we're always looking at the timespan +/-3 minutes around the bolus time. But from my current perspective, it looks like the carbs always come after the bolus.

@burnedikt Yes, on almost every occasion the carbs FOLLOW insulin_bolus. Only on one occasion did I see the carbs first, and then after sometime Diasend seem to do an update as you had observed on occasion.

[widmann]

Here, unfortunately insulin treatments are still not picked up from Diasend most of the time also with latest version 1.1.7. On Diasend with my pump for some reason the insulin is always logged 2-8 min after the carbs. I assume this might be pump specific (Dana-I). However, the problem is not (only) related to matching carbs and insulin as also insulin corrections without carbs are not picked up.

The problem might be related to the entries appearing late at the API? Any hint how to effectively debug the problem would be appreciated.

Thanks a lot!

[burnedikt]

On Diasend with my pump for some reason the insulin is always logged 2-8 min after the carbs. I assume this might be pump specific (Dana-I)

When you say the insulin is logged 2-8 mins later, do you mean the insulin appears within the API response 2-8 mins after the carbs do for the first time but the timestamps (created_at) of the carbs are more or less equal? Or do you mean the difference between the created_at times is 2-8 mins?

Asking, because I am currently working on a version 2 of the bridge which works more reliably in my experience. However, it still has the assumption that the time difference between the carbs' and insulins' created_at dates is no more than 3 minutes (which obviously doesn't hold if it's logged 8 minutes later). Generally, the algorithm could also just match the meal bolus with the "closest" (time-wise) carb record (even if it's 8 minutes earlier) but I am a bit scared this could lead to wrong matchings if multiple meal boli are given in short succession - but now that I've said that, it's probably still better than completely missing out on a matching boli just because it doesn't fall into the timeframe.

However, the problem is not (only) related to matching carbs and insulin as also insulin corrections without carbs are not picked up.

That this also happens for correction boli is weird (since there's no extra logic involved in handling those). Could only imagine that they are pushed to diasend very late and at that time, the timespan to which the bolus was added has already been processed by the bridge. E.g. the bolus happens at 11:00, cam aps fx only checks with the pump every few minutes, so let's say after 10 minutes it gets the confirmation the bolus has been successfully delivered, it then sends this then to diasend (which also happens only every 5 minutes). So in this scenario the bolus at 11 would only appear 15 minutes later on diasend and might be missed out on. In version 2 this wouldn't be an issue anymore as the bridge there does look back further on time than the last 10 minutes. It always looks at all the records since the last synchronized bolus. So boli per se should never be missed.

[widmann]

Or do you mean the difference between the created_at times is 2-8 mins?

Yes indeed, the created_at time stamps differ. Here an example where bolus was not picked up:

    {
        "type": "carb",
        "created_at": "2023-01-22T10:56:52",
        "value": "35",
        "unit": "g",
        "flags": []
    },
    {
        "type": "insulin_bolus",
        "created_at": "2023-01-22T11:00:00",
        "unit": "U",
        "total_value": 6,
        "spike_value": 6,
        "suggested": 7,
        "suggestion_overridden": "yes",
        "suggestion_based_on_bg": "no",
        "suggestion_based_on_carb": "yes",
        "programmed_meal": 7.001,
        "flags": [
            {
                "flag": 1035,
                "description": "Bolus type ezcarb"
            }
        ]
    },

Could only imagine that they are pushed to diasend very late and at that time, the timespan to which the bolus was added has already been processed by the bridge.

Yes, this was my hypothesis. Not sure how to debug this behavior. For example this correction bolus was not picked up.

    {
        "type": "insulin_bolus",
        "created_at": "2023-01-22T13:01:00",
        "unit": "U",
        "total_value": 1,
        "spike_value": 1,
        "suggested": 0,
        "suggestion_overridden": "yes",
        "suggestion_based_on_bg": "no",
        "suggestion_based_on_carb": "no",
        "flags": []
    },

Btw, today I noticed a message on the Diasend frontpage that accounts will be migrated to Glooko in Q1. Would you expect the bridge to also work with Glooko? Otherwise please do not invest any time solving this problem.

Thank you!

[burnedikt]

Btw, today I noticed a message on the Diasend frontpage that accounts will be migrated to Glooko in Q1. Would you expect the bridge to also work with Glooko?

I've noticed that message, too. I'm almost certain the bridge will cease to work with new glooko APIs and I will try to reach out to glooko to get access as I feel like even if glooko and maybe their app is better, the need for having the data on nightscout might still be there.

Could only imagine that they are pushed to diasend very late and at that time, the timespan to which the bolus was added has already been processed by the bridge.

Yes, this was my hypothesis. Not sure how to debug this behavior.

Last time, I simply dumped the api response from diasend for a whole day every minute and could thereby see that the carbs would appear in the api response 5-10 minutes earlier than the bolus. Basically, this breaks the idempotency of the API as two requests with the same parameters (start and end time) will yield different results, depending on when those requests are sent.

[burnedikt]

Should be finally fixed by #38 and Version 2.0.0 of this project. 🎉

[widmann]

Yes, works, great! Reasonable IOB calculations, finally :) Thanks so much for fixing!