Carb treatments disappearing

[motinis]

Describe the bug At about 7:30, I put in a Remote Carb Entry for 16 grams, 2 hour absorption at 7:40 and it appeared as normal (once Loop receives the command it uploads the entry to Nightscout). About 15 minutes later (~7:45) I put in a new Remote Carb Entry for 4 grams, 2 hour absorption at 7:40. The 4 gram entry appeared in the display, but the old 16 gram entry disappeared from the display. I confirmed by going to Reports -> Treatments for the day, and the 16 gram entry does not appear there. Looking at the Day to Day report, the 16 gram entry is also not included in the Total Carbs calculation. COB does take into account the full 20 grams however (likely Loop uploads this too).

To Reproduce I have not tried yet reproducing the error with fictitious entries to Nightscout - I am reporting just as I have seen this issue, and do not have the Loop phone with me to delete the fictitious entries so they won't impact insulin delivery.

Expected behavior Two distinct food entries should appear in Nightscout with the same timestamp.

Your setup information

• Nightscout 15.0.2 on Google Cloud
• Dexcom G6, Bridge. Loop on iPhone SE uploads food entries, insulin, etc.
• Is your issue specific to a browser (Firefox/Safari/Chrome?) or a device (Android phone, etc.)?

Additional context Upgraded 2 days ago from 14.2.6 to 15.0.2. This looks to me related to changes made as part of 14.2.6 --> 15.0.2 as I have not encountered this issue in the past and Loop was not changed in any way.

Update Seen as well now for carbs entered directly in Loop not via a remote entry (both entries). In this case old entry was 5 grams, 1.5 hour absorption at 10:15, and the new entry was 10 grams, 1.5 hours absorption at 10:15 (old entry added somewhere around 7:00 and new entry added at ~9:50). The old entry disappeared from nightscout after the new entry was added in.

[marionbarker]

I am not able to reproduce this.

I tested this with "real" and "test" Loop-dev phones, both are running identical builds (installed from TestFlight), commit 6b7bba6. The real phone reports to NS 15.0.2 site, the test phone reports to NS 14.2.6 site:

First test was 2 different absorptions time, entered one after another.

• time: 9:00 (in the past)
  • carbs 3 g for 3 hour
  • carbs 5 g for 5 hours

Both showed up in NS treatments for NS 14.2.6 ("test" phone) and 15.0.2 (real phone).

Next test - use exactly the same scenario as @motinis (except enter them closer together.

• time: 10:15 (in the future)
  • carbs 5 g for 1.5 hour
  • confirm this shows up in NS
  • carbs 10 g for 1.5 hours
  • both show up

Because I was my "real" phone, which reports to NS 15.0.2, I deleted the fake carb entries as soon as they showed up in Nightscout. I left all the fake entries in place for my test phone.

[motinis]

@marionbarker i wonder if this is somehow related to enough time between entering the data into Loop. Could you please try to wait 15 minutes before creating the second entry?

[motinis]

I am able to consistently reproduce this for Remote Carb entries (both for future entries and past entries) but have not been able to reproduce this when putting the entries directly in Loop.

[motinis]

Interestingly, despite the display changing to match that of the newer entry, it is the older entry that is actually "linked". Changing or deleting the newer entry in Loop has no impact on the treatment in Nightscout. Changing or deleting the older entry in Loop impacts the treatment in Nightscout. Thus for example, if I only delete the older entry, then the treatment is removed from Nightscout (even if the newer entry is still in Loop)

[marionbarker]

This "bug" is not new with 15.0.2. I tested with NS 14.2.6 using remote carb entries and replicated the issue.

• My theory is the "_id" is generated from the "timestamp" as part of the algorithm
• The solution may be to incorporate both "timestamp" and "userEnteredAt" times to generate a unique _id for these remote carbs

Configuration Loop Phone:

• phone SE running iOS 17.1.1
• Loop 3.2.3 (install from TestFlight)
• CGM from NS 14.2.6 test site
• Upload to same site
• Simulated pump

Remote Carb Entry:

• Use LoopCaregiver (download an build fresh today, commit 5704da0)

Test Details with treatment json files

I did the remote entries and then downloaded the treatments from the NS site running 14.2.6. Note the "created_at" time is identical for entries 2 and 3 and they are saved with identical _id. Not sure how _id is generated.

All three carb entries are present on the Loop phone. The COB uploaded from Loop to NS is passed on to LoopCaregiver COB.

For my first entry of remote carbs, I did not modify the time, I used the current time of 08:45 (UTC 13:45), but it was actually entered with current time including seconds:

• First entry of 10 g shows up in NS treatments
  • "_id":"6568920febd0ec3b8f56f4d8"
  • "userEnteredAt":"2023-11-30T13:45:50Z"
  • "created_at":"2023-11-30T13:45:41.000Z"
  • "timestamp":"2023-11-30T13:45:41Z"

Second entry of remote carbs, I modified time to be 08:45 (which it does without any seconds)

• Second entry of 20 g shows up in treatments:
  • "_id":"6568954debd0ec3b8f56f7be"
  • "userEnteredAt":"2023-11-30T13:59:40Z"
  • "created_at":"2023-11-30T13:45:00.000Z"
  • "timestamp":"2023-11-30T13:45:00Z"

Downloaded and saved json file - attached below - used json file for details listed above.

2023-11-30_a_NS_14_2_6_treatments.json

Third entry of remote carbs, I modified time to be 08:45

• Third entry of 30 g shows up in treatments but it has replaced the previous 20 g entry:
  • "_id":"6568954debd0ec3b8f56f7be"
  • "userEnteredAt":"2023-11-30T14:28:21Z"
  • "created_at":"2023-11-30T13:45:00.000Z"
  • "timestamp":"2023-11-30T13:45:00Z"

Downloaded and saved json file - attached below - used json file for details listed above.

2023-11-30_b_3-entries_NS_14_2_6_treatments.json

[motinis]

My theory is the "_id" is generated from the "timestamp" as part of the algorithm The solution may be to incorporate both "timestamp" and "userEnteredAt" times to generate a unique _id for these remote carbs

is there any reason not to use a random uuid for the _id? I’m really not familiar with the data model here though

[marionbarker]

I repeated the test, this time I exported the critical logs from Loop. The unique "syncIdentifier" from Loop is found in the NS treatment json, but not the uuid.

I do not know where the "_id" in the Nightscout treatment json is generated - if it is Nightscout, then it is a cgm-remote monitor issue. If it is Loop then it is a Loop issue.

Test Details

• Enter remote carbs:
  • use current time (so will include seconds), 30 g at 13:33 (18:33 UTC)
  • select 13:33 and enter 20 g
  • select 13:33 and enter 10 g (loop has all 3, NS has only 30 and 10)
    • save NS treatments download
    • 2023-11-30_18-30_NS-treatments-remote-entry-3.json
  • select 13:33 and enter 5 g (loop has all 4, NS has only 30 and 5)
    • save NS treatments download
    • 2023-11-30_18-30_NS-treatments-remote-entry-4.json
    • post Loop carb.json file after all 4 entries: Carbs_remote_entry_4.json

Note - exported critical logs from Loop between each remote entry. Each carb.json was same as one before but with added entry. Therefor only post the final carb.json.

[bewest]

Please ensure that carboyhydrate and other events get entered with unique created_at timestamps. If the exact same timestamp is sent, we expect upsert operation to update the existing record, rather than produce two identical records. If iAPS is zeroing out part of the time for unique events, this could lead to updating records rather than inserting a new record.

[ps2]

This behavior in NS is pretty unfortunate. Nightscout does implement separate POST (create) and PUT (update) operations like a rest api should, and it has an _id system that can be used to identify records accurately for updates. So all the parts are there for a standard api. But this behavior of treating a POST like an update for any matching timestamps breaks things. Add on top of that that the created_at field carrying double duty as representing both the time of entry and the time of the meal leaves clients in a bind as to how to represent two entries with the same meal time, and leads to ugly hacks like this: https://github.com/LoopKit/NightscoutService/pull/10

@bewest Is there any hope for an API update that would allow clients to manage this in a sane way? Like requiring updates use "PUT"? Or creating a new "POST" endpoint that actually just does create, and does not do upserts.

[bewest]

Sure, @ps2. There is the v3 API which is oriented toward servicing modern AID systems. It has a POST for creating records and a PUT and PATCH for updating them. In addition, there have been discussions for redesigning a profile API to help with Loop/AAPS and remote commands. If v3 API doesn't seem feasible it's certainly worth discussing patching the v1 or coming up with another API. Another major item for work is accuracy and reproducible reports. There's a lot of work to do and very few resources... The testing framework and most tests need to be rewritten without benv/jsdom, not to mention resuming the mongo changes. :shrug: