Multi import of “slowly absorbed meal”

[funkstille]

slowly absorbed meal carbohydrates somehow sometimes got imported multiple times. I could not figure out so far when and why it happens. See pictures:

See pictures:

[burnedikt]

oh wow ... that shouldn't happen. Which version of the bridge are you running on? There was a bug in versions 0.7.1 to 0.7.4 where it could happen that events were imported multiple times. Never seen anything like this before though. Seems like in each iteration, it imports all of the past events again.

Usually, the loop is designed to only load events between the last known event (+1s) and the current time which should prevent processing events multiple times.

[burnedikt]

Did the duplication of events stop after the 0.6 correction bolus? Because that would probably mean that the slowly absorbed meal events were for some reason not considered as standard boli and therefore the algorithm always started the search for events after the last "normal" bolus or carb correction.

[funkstille]

Looks like it’s a problem with the webspace / Script / something else. all carbs got multiplied

[funkstille]

I am using a forg of your latest master CGM-Repo. Problem still occurs, this morning only for the second Bolus:

nightscout acts unresponsive.

I have switched the hosting service from heroku to railway. but for the first 2 days there has been no issue… will investigate

[funkstille]

in the database there are multible entries like:

_id
6358ccd89ce7b1cef78fea6f
eventType
"Meal Bolus"
insulin
0.6
carbs
10
app
"diasend"
date
1666761660000
device
"KidsAP Pump (TCM00092FB_2022410536)"
created_at
"2022-10-26T05:59:52.309Z"
utcOffset
120

_id
6358ce439ce7b1cef79065d2
eventType
"Meal Bolus"
insulin
0.6
carbs
10
app
"diasend"
date
1666761660000
device
"KidsAP Pump (TCM00092FB_2022410536)"
created_at
"2022-10-26T06:05:55.806Z"
utcOffset
120

[funkstille]

This morning I cleared all treatments in mongo db, since nightscout website was not able to reload. Afterwards I redeployed and everything was running smooth. A meal bolus worked fine, but then another "slowly absorbed meal" Bolus was added and it started again.

Eyery 4 minutes 2-3 treatment entries are pushed ito the database. So the multible entries are growing eveytime the diasend script ist checking for new values and treatments. As already pointed out before, this behavior started since I moved to the railway stack.

[burnedikt]

I've also seen this issue yesterday with a "normal" bolus - not really sure what's going on there. Will take a deeper look in the upcoming days. But for me, this happened in heroku so I don't think it's an exclusive Railway thing. But it definitely appears to only happen under certain conditions. Sorry for the troubles there. For now, you might also be able to downgrade to an older version, like e.g. 0.7.0. In that version, some events may go missing because the loops for CGM and other events like boli or carbs are not separate yet but likely this also prevents them from being imported multiple times.

[funkstille]

My two cents (but I would not call myself a developer at all):

If I compare the datasets from diasend api for type: carb ("slowly absorbed meal) it has a created at timestamp that is converted into an epoch ts as attribute in nightscout / mongo "date" field. But there is also the "created at" field in nightscout / mongodb which gets a slighty different value every time (I guess) the importer sript runs again.

Dublicated dataset:

_id 6363b3f09ce7b1cef73fee26
eventType "Carb Correction"
carbs 3
app "diasend"
date 1667478400000
device "KidsAP Pump (XYZ)"
created_at "2022-11-03T12:28:32.236Z"
utcOffset 60

_id 6363b4e39ce7b1cef7403f56
eventType Carb Correction
carbs 3
app diasend
date 1667478400000
device KidsAP Pump (XYZ)
created_at 2022-11-03T12:32:35.260Z
utcOffset 60

Wouldnt it be better to use the same TS from diasend for both "created at" and "date" in mongo db? So duplikates would be overriden automatically ?

[burnedikt]

seems like this is happening in particular for snacks and slowly absorbed meal entries.

[burnedikt]

Wouldnt it be better to use the same TS from diasend for both "created at" and "date" in mongo db? So duplikates would be overriden automatically ?

As far as I understood nightscout, it never overrides entries but always creates new ones. If you want to override an entry, you need to manually delete and recreate it.

[burnedikt]

The multiple import of events should be solved with version 0.7.7.

[burnedikt]

Ran this for a few days now for testing purposes and couldn't reproduce the multi-import issue anymore. Closing this issue. Feel free to reopen if you still encounter issues.