kashamalasha / nightscout-widget-electron

An Electron widget application to show on a desktop blood glucose value is getting from Nightscout API
https://github.com/kashamalasha/nightscout-widget-electron
GNU General Public License v3.0
34 stars 7 forks source link

Stopped working #93

Closed j-kaltes closed 4 months ago

j-kaltes commented 10 months ago

After successfully running Owlet under Linux, Owlet stopped working. Changing to ~/.config/Owlet, this problem continued after rm -rf 'Code Cache' OwletCache GPUCache

Only after rm -rf SingletonSocket .org.chromium.Chromium.5osK3X /tmp/scoped_diry6XhxF did the problem subside.

kashamalasha commented 10 months ago

Hi! Thank you for the feedback.

Could you specify the following attributes:

j-kaltes commented 10 months ago

Owlet doesn’t work with at all with nightscout server I have: "version":"15.0.2", "apiVersion":"3.0.3-alpha" This is because Owlet uses the command: api/v3/entries?sort$desc=date&limit=6&fields=sgv,direction,srvCreated A bug in Nightscout makes that it doesn’t work with when limit starts with a digit larger than 4, so it works with limit=10, but not limit=6 With limit=6, I receive the answer: { "status": 400, "message": "Parameter limit out of tolerance" } Why do you use v3? Everyone else uses v1.

Instead of this, I use Juggluco server running on my computer as Nightscout server. Juggluco server receives its data from Juggluco on the phone. See: https://www.juggluco.nl/Juggluco/webserver.html

After the update to Owlet-0.8.0-beta-linux-x86_64.AppImage, it still has this problem.

It happens when I suspend the computer and the next day, I resume it. At that moment Juggluco server hasn’t received the data from the phone jet. Owlet does the following commands: /api/v2/authorization/request/${TOKEN} Which returns in this case: {"token":"JZ-gTBnnJ86VviDI53qU82Ii5BHUNi8q4niMIohCmDhtuixBh1XbrQLbCTwrq7XeIG6N9dNiYCimN5C2pJJx2j7sKFa4UZ-E7cc0PAvc5khDXZpqFhqQ-faefhnjQ1e5gqHTeCUqZJ0lAWz1YTJcItYnkz.9CBBc5RxHH6VoJKW0vXrAg8F","sub":"Juggluco","permissionGroups":[["::read"],["::read"]],"iat":1706437740,"exp":1706452140}

Thereafter Owlet asks: /api/v3/entries?sort%24desc=date&limit=6&fields=sgv%2Cdirection%2CsrvCreated&type%24eq=sgv And receives that data from the previous day: {"status":200,"result":[{"sgv":98,"direction":"Flat","srvCreated":1706390192000},{"sgv":99,"direction":"Flat","srvCreated":1706390132000},{"sgv":101,"direction":"Flat","srvCreated":1706390072000}, …. Thereafter Owlet doesn’t contact the server anymore for a very long time.

By the way, why use srvCreated and not date? date is specified by the uploader, srvCreated is created by Nightscout itself is not equal to the measurement time.

kashamalasha commented 10 months ago

Thank you for the investigation.

I chose the v3 API because it is well-structured and returns much less data when using path params filtering compared to v1 and v2.

An interesting point to consider is the limit parameter. I haven't encountered this issue before, possibly due to a bug in some Nightscout latest releases. I use the limit=6 statement because of the trend calculation function, which relies on the last 6 measurements for the last 30 minutes based on the Abbott Freestyle Libre algorithm. Proper calculation requires at least six recent measurements.

https://github.com/kashamalasha/nightscout-widget-electron/blob/1da6038ea7b4778b8dfb19f2e15d1d369b6d4f67/js/util.js#L76

However, in general, we only need the two latest measurements to calculate the delta between them.

An interesting point is using date instead of srvCreated. I overlooked that srvCreated might be missing for old devices that could still contribute data to Nightscout. This is due to its description:

 srvCreated : integer($int64) 
    example : 1525383610088
description : The  server's timestamp of document insertion into the database (Unix epoch  in ms). 
              This field appears only for documents which were inserted by API  v3. 
              Note: this field is immutable by the client (it cannot be updated or patched)

I will try substituting srvCreated with the date field and test it.

j-kaltes commented 10 months ago

srvCreated is present, but as I said, it doesn’t give the measurements time. When I turn on my computer the next day, the glucose measurements of the past night all get a srvCreated time of the time they are added to the Nightscout database. Is that what you need to know? This is an example of an item of the night:

{
      "app": "Juggluco",
      "device": "3MH00V0LR7D",
      "date": 1706498381000,
      "sgv": 115,
      "delta": 0.7,
      "direction": "Flat",
      "type": "sgv",
      "utcOffset": 0,
      "identifier": "37524c30-5630-3048-44ee-10864eeeeeee",
      "created_at": "2024-01-29T03:19:41.000Z",
      "srvModified": 1706515511271,
      "srvCreated": 1706515511271
    },

1706515511271 milliseconds since 1 Jan 1970 00:00 GMT, is Mon Jan 29 08:05:11 UTC 2024 that is very different from the time of measurement, which is given with "created_at" and “date”. EDIT: The v3 interface is new, used by nobody and full of bugs. If you want to get less information you can also use: pebble?count=6&units=mg This gives something like:

{
  "status": [
    {
      "now": 1706528632271
    }
  ],
  "bgs": [
    {
      "sgv": "80",
      "trend": 4,
      "direction": "Flat",
      "datetime": 1706528535000,
      "bgdelta": 0,
      "iob": "0",
      "bwp": "0",
      "bwpo": 4.4,
      "cob": 0
    },
    {
      "sgv": "82",
      "trend": 4,
      "direction": "Flat",
      "datetime": 1706528475000
    },
    {
      "sgv": "83",
      "trend": 4,
      "direction": "Flat",
      "datetime": 1706528415000
    },
    {
      "sgv": "83",
      "trend": 4,
      "direction": "Flat",
      "datetime": 1706528354000
    },
    {
      "sgv": "83",
      "trend": 4,
      "direction": "Flat",
      "datetime": 1706528294000
    },
    {
      "sgv": "82",
      "trend": 4,
      "direction": "Flat",
      "datetime": 1706528234000
    }
  ],
  "cals": []
}

This interface is very old, only units=mg is somewhat newer. To be robust, you can better test on the size of glucose values. If they are less than 35, they are mmol/L otherwise mg/dL. You can use the pebble interface even with the webservice in xDrip.

kashamalasha commented 10 months ago

I can not reproduce the bug with the limit=6 param on my NS.

image

But I got the previous version on my site

{
  "status": 200,
  "result": {
    "version": "14.2.6",
    "apiVersion": "3.0.3-alpha",
    "srvDate": 1706530606535,
    "storage": {
      "storage": "mongodb",
      "version": "6.0.8"
    }
  }
}

If you wouldn't mind, could you please share your NS URL and API key in a direct message via e-mail: dmitry.burnyshev@gmail.com with me to help reproduce the bug?

j-kaltes commented 10 months ago

https://nightscout.blubber.dk/api/v3/entries?sort$desc=date&limit=6

https://nightscout.blubber.dk/api/v3/entries?sort$desc=date&limit=10

j-kaltes commented 10 months ago

I downloaded the last version from https://github.com/nightscout/cgm-remote-monitor: 15.0.2 (as I already wrote above: https://github.com/kashamalasha/nightscout-widget-electron/issues/93#issuecomment-1913734949). You use 14.2.6. https://nightscout.blubber.dk/status returns:

{
  "status": 200,
  "result": {
    "version": "15.0.2",
    "apiVersion": "3.0.3-alpha",
    "srvDate": 1706535337163,
    "storage": {
      "storage": "mongodb",
      "version": "6.0.4"
    },
    "apiPermissions": {
      "devicestatus": "crud",
      "entries": "crud",
      "food": "crud",
      "profile": "crud",
      "settings": "crud",
      "treatments": "crud"
    }
  }
}
kashamalasha commented 10 months ago

@j-kaltes Could you check the functionality with the latest release, please? https://github.com/kashamalasha/nightscout-widget-electron/releases/tag/v0.8.2-beta

Thanks in advance

kashamalasha commented 10 months ago

@j-kaltes Could you update the status? Thanks in advance.

j-kaltes commented 10 months ago

Yes, it now works with Nightscout 15.0.2. But that was not my interest. I don’t use Nightscout. I had the problem that Owlet stopped working when using it with the Nightscout server in Juggluco. Nothing is changed in that regard. Some of the situations when problems arise:

  1. Killing Owlet with “kill pid” instead of stopping it with (x) . Sometimes you can’t start it anymore thereafter. One time it gave this error message: [373476:0130/180214.406278:ERROR:bus.cc(407)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix") At other times it immediately ends without a message.
  2. Killing the server also gave problems when the authentication token isn’t saved by the server. Owlet doesn’t ask again for the generation of a new token. In the new version the web server in Juggluco saves this token, but should it?
kashamalasha commented 10 months ago

What is Juggluco? I found on the Internet that is an Android app. What do you mean when saying "using it with the Nightscout server in Juggluco"? Was this server hosted on the web or locally? What kind of Linux distro do you use? Send me a log pls. Thanks.

j-kaltes commented 10 months ago

Juggluco is an Android app that can receive glucose values from Freestyle Libre 2 and 3 sensors via Bluetooth. It contains a web server that implements only the subset of Nightscout commands that are really used by Nightscout clients. This webserver in Juggluco can be used by all Nightscout clients as far as I know (for Aapsclient you have to use the next version of Juggluco). The network part of Juggluco is also put in a command line program called Juggluco server this can be used as a gateway between phones when they can’t be connected directly, but also contains the Nightscout web server. The relevant logs I showed already. The most important thing is that Owlet doesn’t connect anymore. Thus the logs show nothing anymore. This is another variant:

2024-01-30-11:07:52 313677 webserver: watchcommands len=514 GET /api/v2/authorization/request/Dit7Moet8Je9Weten HTTP/1.1^M {"token":"frzf1.2Y4Bf8afVcbcnKfj0.sHUVSb-67TBaNVmRXaZgMDYkSl6vIYyGVpiTIJelF1LZwLUAtCtOlOlB-SZXSFYB-L6DigcTNSlTg51cZrCQlIvzMONsbOkVhQD53bsFtpWwsf3CeP3mQZgHNnuh0Mq.PpFXCjHA84metFUBqTnZx5zQbIL","sub":"Juggluco","permissionGroups":[["::read"],["::read"]],"iat":1706609272,"exp":1706623672} 2024-01-30-11:07:52 313678 webserver: watchcommands len=711 GET /api/v2/entries/sgv?count=10 HTTP/1.1^M [{"_id":"3MH00V0LR7D#12200","device":"Juggluco","dateString":"2024-01-30T01:42:28.000Z","sysTime":"2024-01-30T01:42:28.000Z","date":1706578948000,"sgv":90,"delta":1.400,"direction":"Flat","noise":1,"filtered":90000,"unfiltered":90000,"rssi":100,"type":"sgv","units_hint":"mgdl"}, 2024-01-30-11:07:55 314065 webserver: watchcommands len=755 GET /api/v3/entries?sort%24desc=date&limit=10&fields=sgv%2Cdirection%2Cdate&type%24eq=sgv HTTP/1.1^M {"status":200,"result":[{"date":1706578948000,"sgv":90,"direction":"Flat"},{"date":1706578887000,"sgv":88,"direction":"Flat"},{"date":1706578827000,"sgv":86,"direction":"Flat"},{"date":1706578766000,"sgv":85,"direction":"Flat"},

Nothing happens hereafter.

Here another time: 2024-01-31-09:37:48 377172 webserver: watchcommands len=514 GET /api/v2/authorization/request/Dit7Moet8Je9Weten HTTP/1.1^M {"token":"J01yseLwHvdFa8gEmPCUbuEaQseCLsTOJ8hQ51xxkjHMYTbqa1kV9sv20A4JDsiHaTNOCyLAeCxpof6Z7RYy9rP9EYAdwvLf4WBM8YTFxtTfDiNEbSMts2gqozX92YQ1s73AHSYVsUfYs8ytRpDYZE4gNjEKCyHSNibM-wC20N6abQbcj1c","sub":"Juggluco","permissionGroups":[["::read"],["::read"]],"iat":1706690268,"exp":1706704668} 2024-01-31-09:37:48 377173 webserver: watchcommands len=839 GET /api/v3/entries?sort%24desc=date&limit=10&fields=sgv%2Cdirection%2Cdate&type%24eq=sgv HTTP/1.1^M {"status":200,"result":[{"date":1706635086000,"sgv":122,"direction":"FortyFiveUp"},{"date":1706635026000,"sgv":119,"direction":"FortyFiveUp"},{"date":1706634965000,"sgv":116,"direction":"FortyFiveUp"},{"date":1706634904000,"sgv":113,"directio1706690268 377173 webserver: sock=11 sendall len=728 2024-01-31-09:46:34 381571 webserver: watchcommands len=514 GET /api/v2/authorization/request/Dit7Moet8Je9Weten HTTP/1.1^M {"token":"nKsdOa5VCgBQjp64qhklTEwItel6MxY13fTN-c34-7iknZXy9FvWY8TF4Q9bXOG3hmoonEE.vIxsXhR.ZSbOp5zwA8Bmm60DLGbkNyV.ZyEnHbLh0qGxZGmpyGGNRQNa475JWOgIt4iczDwdeEmxTdl-CbOAkWEMOtItPgk5UklcV4epxcZ","sub":"Juggluco","permissionGroups":[["::read"],["::read"]],"iat":1706690794,"exp":1706705194}

kashamalasha commented 10 months ago

Please send me the main.log file from .config folder. If the app cannot connect to the server it should trigger an networkConnectionError after 5 failed tries to get the data.

kashamalasha commented 10 months ago

@j-kaltes Could you send me a main.log file? I need to check the output of the application, not the webserver's logs. Thanks.

j-kaltes commented 10 months ago

main.log shows nothing. Today I had a few times the same problem. Pressing Test or Save helped nothing. Restarting didn't help. What helped was:

rm -rf ~/.config/Owlet/*Cache*

Now I have the same problem. When I use another host name for the same site, the problem was solved. Also when I change back to the previous host name. It is 16:42 +01:00 and the log ends with the following:

[2024-01-29 11:12:14.720] [warn] Renderer: Settings were updated. Widget will be restarted. color: orange [2024-01-29 11:16:02.510] [warn] Renderer: Settings were updated. Widget will be restarted. color: orange [2024-01-29 11:17:33.147] [warn] Renderer: Settings were updated. Widget will be restarted. color: orange [2024-01-29 11:17:43.648] [warn] Renderer: Settings were updated. Widget will be restarted. color: orange [2024-01-29 11:18:35.218] [error] Renderer: Request status: 400 - Bad Request: Parameter limit out of tolerance - was encountered over than 5 times color: red [2024-01-29 11:19:12.528] [warn] Renderer: Settings were updated. Widget will be restarted. color: orange [2024-01-29 15:57:23.212] [warn] Renderer: Settings were updated. Widget will be restarted. color: orange [2024-01-29 15:58:14.898] [error] Renderer: Request status: 400 - Bad Request: Parameter limit out of tolerance - was encountered over than 5 times color: red [2024-01-30 15:14:23.961] [error] Renderer: The server is not responding. Check your nightscout site address. - was encountered over than 5 times color: red [2024-02-01 12:32:55.314] [warn] Renderer: Settings were updated. Widget will be restarted. color: orange [2024-02-01 20:16:41.479] [warn] Renderer: Settings were updated. Widget will be restarted. color: orange [2024-02-02 11:09:02.452] [warn] Renderer: Settings were updated. Widget will be restarted. color: orange [2024-02-02 11:22:02.180] [warn] Renderer: Settings were updated. Widget will be restarted. color: orange [2024-02-02 11:35:38.146] [warn] Renderer: Settings were updated. Widget will be restarted. color: orange [2024-02-02 16:37:40.386] [warn] Renderer: Settings were updated. Widget will be restarted. color: orange [2024-02-02 16:39:49.560] [warn] Renderer: Settings were updated. Widget will be restarted. color: orange [2024-02-02 16:40:30.621] [warn] Renderer: Settings were updated. Widget will be restarted. color: orange

kashamalasha commented 10 months ago

@j-kaltes Could you provide me a reachable web server for test with your env set up? I can not reproduce this kind of behavior on my Linux machine. Thanks.

j-kaltes commented 10 months ago

I have send an URL and token by e-mail. It will be down most of the time, the problem arises after a down time. EDIT: Further settings: Request interval: 10 seconds Stale Data: 3 Minutes Show data age: x Units in MMOL/L: x CALC Trend: not set