search

NightscoutFoundation / xDrip

Nightscout version of xDrip+
https://jamorham.github.io/#xdrip-plus
GNU General Public License v3.0
1.39k stars 1.14k forks source link

xdrip uploads bad libre data to nightscout on some rare events. #2763

Open tzachi-dar opened 1 year ago

tzachi-dar commented 1 year ago

Here is the nightscout data image

As one can see, there is a point that does not seem related to the existing points in graph. Here is the relevant data from nightscout at that time:

{"_id":"642996b2a8c5f5a37dd6e96a","device":"xDrip-LimiTTerWifi","date":1680447153395,"dateString":"2023-04-02T14:52:33.395Z","sgv":203,"delta":10.891,"direction":"SingleUp","type":"sgv","filtered":223529.395,"unfiltered":223529.395,"rssi":100,"noise":1,"sysTime":"2023-04-02T14:52:33.395Z","utcOffset":180,"mills":1680447153395},
{"_id":"6429958ea8c5f5a37dd6e797","device":"xDrip-LimiTTerWifi","date":1680446855240,"dateString":"2023-04-02T14:47:35.240Z","sgv":192,"delta":13.782,"direction":"SingleUp","type":"sgv","filtered":212705.8664,"unfiltered":212705.8664,"rssi":100,"noise":1,"sysTime":"2023-04-02T14:47:35.240Z","utcOffset":180,"mills":1680446855240},
{"_id":"64299588a8c5f5a37dd6e783","device":"xDrip-LimiTTerWifi","date":1680446435240,"dateString":"2023-04-02T14:40:35.240Z","sgv":173,"delta":13.782,"direction":"FortyFiveUp","type":"sgv","filtered":193411.75019999998,"unfiltered":193411.75019999998,"rssi":100,"noise":1,"sysTime":"2023-04-02T14:40:35.240Z","utcOffset":180,"mills":1680446435240},
{"_id":"64299588a8c5f5a37dd6e781","device":"xDrip-LimiTTerWifi","date":1680446135240,"dateString":"2023-04-02T14:35:35.240Z","sgv":165,"delta":13.782,"direction":"Flat","type":"sgv","filtered":185176.45669999998,"unfiltered":185176.45669999998,"rssi":100,"noise":1,"sysTime":"2023-04-02T14:35:35.240Z","utcOffset":180,"mills":1680446135240},
{"_id":"642991ada8c5f5a37dd6df36","device":"xDrip-LimiTTerWifi","date":1680445863975,"dateString":"2023-04-02T14:31:03.975Z","sgv":161,"delta":0.98,"direction":"Flat","type":"sgv","filtered":181882.3393,"unfiltered":181882.3393,"rssi":100,"noise":1,"sysTime":"2023-04-02T14:31:03.975Z","utcOffset":180,"mills":1680445863975},

{"device":"xDrip-LimiTTerWifi","date":1680445595240,"dateString":"2023-04-02T14:26:35.240Z","sgv":50,"delta":13.782,"direction":"Flat","type":"sgv","filtered":70705.87705,"unfiltered":70705.87705,"rssi":100,"noise":0,"sysTime":"2023-04-02T14:26:35.240Z","utcOffset":180,"_id":"6429958ea8c5f5a37dd6e799","mills":1680445595240},

{"_id":"642991a9a8c5f5a37dd6df31","device":"xDrip-LimiTTerWifi","date":1680445323975,"dateString":"2023-04-02T14:22:03.975Z","sgv":160,"delta":0.98,"direction":"FortyFiveUp","type":"sgv","filtered":180117.63355,"unfiltered":180117.63355,"rssi":100,"noise":1,"sysTime":"2023-04-02T14:22:03.975Z","utcOffset":180,"mills":1680445323975},
{"_id":"64298e7ca8c5f5a37dd6d8d2","device":"xDrip-LimiTTerWifi","date":1680445051282,"dateString":"2023-04-02T14:17:31.282Z","sgv":153,"delta":3.529,"direction":"Flat","type":"sgv","filtered":173999.98695,"unfiltered":173999.98695,"rssi":100,"noise":1,"sysTime":"2023-04-02T14:17:31.282Z","utcOffset":180,"mills":1680445051282},
{"_id":"64298e7ca8c5f5a37dd6d8d0","device":"xDrip-LimiTTerWifi","date":1680444631282,"dateString":"2023-04-02T14:10:31.282Z","sgv":149,"delta":3.529,"direction":"Flat","type":"sgv","filtered":169058.81084999998,"unfiltered":169058.81084999998,"rssi":100,"noise":1,"sysTime":"2023-04-02T14:10:31.282Z","utcOffset":180,"mills":1680444631282},
{"_id":"64298b68a8c5f5a37dd6d2a5","device":"xDrip-LimiTTerWifi","date":1680444263173,"dateString":"2023-04-02T14:04:23.173Z","sgv":143,"delta":2.353,"direction":"Flat","type":"sgv","filtered":163294.1054,"unfiltered":163294.1054,"rssi":100,"noise":1,"sysTime":"2023-04-02T14:04:23.173Z","utcOffset":180,"mills":1680444263173},

on 17:26:35.240 we have the bad data. Here is the bg data on xDrip (using a query on the bg table): 2023-04-02 17:10:31 1680444631282 149.05881085 2023-04-02 17:17:31 1680445051282 153.99998695 2023-04-02 17:22:03 1680445323975 160.11763355 2023-04-02 17:26:35 1680445595240 50.70587705 ** 2023-04-02 17:31:03 1680445863975 161.8823393 2023-04-02 17:35:35 1680446135240 165.1764567 2023-04-02 17:40:35 1680446435240 173.4117502 2023-04-02 17:47:35 1680446855240 192.7058664 2023-04-02 17:52:33 1680447153395 203.529395 2023-04-02 17:57:35 1680447455042 217.8823351 2023-04-02 18:02:33 1680447753808 225.99998155

But the point does not show on xDrip screen or bgtable

Looking at the xdrip logs, I believe the relevant points are:

2023-04-02 17:17:31 LibreOOPAlgorithm Creating BG time = 13141 raw = 1479 2023-04-02 17:18:29 LibreOOPAlgorithm Creating BG time = 13142 raw = 1494 2023-04-02 17:31:33 LibreOOPAlgorithm Creating BG time = 13155 raw = 0 2023-04-02 17:47:35 LibreOOPAlgorithm Creating BG time = 13171 raw = 1808 2023-04-02 17:48:32 LibreOOPAlgorithm Creating BG time = 13172 raw = 1831

The problematic point seems where raw ==0 and the bad reading probably came from 17:47:35 (exact number of seconds and milli seconds match)

Here are the libreblock timing points: 2023-04-02 17:04:23 1680444263173 B702C818030000000000000000000000000000000000000000D8070B6205002C1E80580500981E80530500141E805405008C5D00670500785D006A0500685D006C0500485D00E90400745D00FF0400685D000505008C5D00140500CC5D00150500085E00280500C05C003305087A1D80460500B41D804B0500B41D805D0300E05B00190300409A00AA0300C89A00890300785B006C0300889B009B0300605C00F00300649800F90300CC1E80540400041F80D40400801E80600500785D002B0300EC1601480300DC160124030064D7002E0300281701610300C8D700CD0300CC99000F0400D49A00CA0400009B00710500549B00D90500A49A00CC05004C9B006305009C9A00620500289B00270500DC9A006B0400F09B00F10300B49B008C0300089C007D0300C89B003A0400049C003B0400A85B00D90300A09B00483301007ABD30089A0D4F51140396805A00CDA60E5F1A0000B1586A 2023-04-02 17:17:31 1680445051282 C745CD05A445DE05A2C5E981A285E6819D45EC817B05D3056A85D6056085D705D404E8815444F0815533114A 2023-04-02 17:18:29 1680445109000 D6C5D005B445D4059705E181A185E181A285E6818605E0816C85D405B545D4056085D705D404E8815633919E 2023-04-02 17:31:03 1680445863975 B702C8180300000000000000000000000000000000000000FE99020C070600C85C000A0600A05C00A40500E45D00B40500445D00C70500D45C00D605000C5D00E405003C5D00F40500445D00FB0500405D00FD0500385D00020600985D00030600885D00040600205D00030600F85C000606007C5D00060600245D005D0300E05B00190300409A00AA0300C89A00890300785B006C0300889B009B0300605C00F00300649800F90300CC1E80540400041F80D40400801E80600500785D00B50500445D00480300DC160124030064D7002E0300281701610300C8D700CD0300CC99000F0400D49A00CA0400009B00710500549B00D90500A49A00CC05004C9B006305009C9A00620500289B00270500DC9A006B0400F09B00F10300B49B008C0300089C007D0300C89B003A0400049C003B0400A85B00D90300A09B00633301007ABD30089A0D4F51140396805A00CDA60E5F1A0000B1586A 2023-04-02 17:31:32 1680445892970 000008040786CC0506C6D7050406D2050386D805E4C5D305B445D405B545D4056085D705D404E8816333E3E3 2023-04-02 17:47:35 1680446855240 10C7A309EA46AC09A486B7058446B8096C86BA0926C6C2091246C50500000804B545D4056085D70573334741

tzachi-dar commented 1 year ago

The strange thing is that the problematic point was not sent to the followers, although sent to nightscout and was on main device. In any case, it seems that the issue is that parsing ble history brings a data of zero.

data = fromHex('10C7A309EA46AC09A486B7058446B8096C86BA0926C6C2091246C50500000804B545D4056085D70573334741')
console.log(parseBLEData(data))
[LOG]: {
  "glucose": [
    {
      "value": 1808,
      "temperature": 6716,
      "temperatureAdjustment": 8
    },
    {
      "value": 1770,
      "temperature": 6852,
      "temperatureAdjustment": 8
    },
    {
      "value": 1700,
      "temperature": 7032,
      "temperatureAdjustment": 4
    },
    {
      "value": 1668,
      "temperature": 7044,
      "temperatureAdjustment": 8
    },
    {
      "value": 1644,
      "temperature": 7080,
      "temperatureAdjustment": 8
    },
    {
      "value": 1574,
      "temperature": 7212,
      "temperatureAdjustment": 8
    },
    {
      "value": 1554,
      "temperature": 7252,
      "temperatureAdjustment": 4
    },
    {
      "value": 0,
      "temperature": 128,
      "temperatureAdjustment": 4
    },
    {
      "value": 1461,
      "temperature": 7492,
      "temperatureAdjustment": 4
    },
    {
      "value": 1376,
      "temperature": 7544,
      "temperatureAdjustment": 4
    }
  ],
  "wearTimeMinutes": 13171,
  "crc": 16711
}

We can see here a value with zero. The condition value of readingData.ClearErrors seems wrong for history:

Iterator<GlucoseData> it = history.iterator();

while (it.hasNext()) {

    GlucoseData glucoseData = it.next();

    if (libreTrendPoints.get(glucoseData.sensorTime).isError()) {

        it.remove();

    }

}

That is we are checking for the raw values which might not even exist for this specific point. We should do the comparison against the current point as well.