nightscout / Trio

MIT License
45 stars 125 forks source link

[Bug] Log files grow excessively large #272

Open LiroyvH opened 1 month ago

LiroyvH commented 1 month ago

Probably a known issue, but I couldn't find any open (or closed for that matter) issue about it so wanted to log (lol) it anyway:

Describe the bug

The log files grow very very large. Its approximately 35MB of data per day, mostly due to repetitive messages (with the exact same content over and over and over again), such as (note: excerpts from a logfile that has 24H of data in it):

grep "HealthKit Store" log_prev\ 2.txt | wc -l 96258 (occurrences)

grep "Stored carb entry" log_prev\ 2.txt| wc -l 10286 (occurrences)

grep createContext log_prev\ 2.txt | wc -l 27656 (occurrences)

The latter can be filtered for repetitive (in quick succession) lines related to createContext such as these:

2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x

grep "JavaScript log: x" log_prev\ 2.txt | wc -l 11337 (occurrences)

Logging a lot isn't necessarily an issue and 35MB isn't the end of the world in terms of storage, but it makes the logs much harder to read, there's insane amounts of repetitive/redundant data in there and this likely leads to increased battery usage. I hope this can be improved.

Also to put it in perspective: there's a total of roughly 200K lines in the log file. That means the repetitive "HealthKit Store" lines make up for nearly 50% of all the information in the log.

Attach a Log

Excerpts provided above.

To Reproduce

Use the app I guess

Expected behavior

More reasonable logging. Logging the same stuff over and over and over again is like a constant debug mode, not sure if that's useful. But in any case there's low hanging fruit such as the weird OpenAPS messages just containing x, + or - and that thousands and thousands of times. Incidentally, the logs saying it logged x amount of carbs to HealthKit seems to suggest that its sending the same carb entries to HealthKit over and over and over again. I'm not sure if that's useful and likely leads to unnecessary battery usage.

Setup Information (please complete the following information):

Smartphone:

Pump:

CGM:

dnzxy commented 1 month ago

Just to add some context to this:

grep createContext log_prev\ 2.txt | wc -l 27656 (occurrences)

The latter can be filtered for repetitive (in quick succession) lines related to createContext such as these:

2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x
2024-06-03T18:24:44+0200 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: x

These are actually mis-formatted (or rather printed) logs of the autosens algorithm doing its thing. A formatted way of this would look something more like this, although this was just a quick fiddle and still not properly formatted:

image

That's autosens going through all the provided minutes and "encoding" what each glucose value is attributed to.

You can now ask "Why is this even necessary to be logged?" or "Shouldn't this only be logged when you are only using AS and not relying on DynamicISF". The current DynISF implementation in Trio (and iAPS) does hijack the AS results though; it uses it as a basis for setting its dynamic factor to then calculate a new, adjusted ISF (and CR, if enabled).

This still needs work and logs are messy and overcrowded and there are redundant logs in there. Just wanted to put this specific portion of 100s of - x + log entries into context.