Fix: Aggregate Logs and formatting

mkellerman commented 1 week ago

This is my first attempt at swift language. Any criticism welcome!

The formating of javascript console log is not formatted in a way that it is easy for others to read, or created automation around in parsing it. I suggestion here is to us as minimal code as possible to format the logs without it being a word to word regex replacement.

This can be tweaked and tailored, and some 'fixed' values could be addressed in a seperate method, but this is my first implementation.

This should also reduce the size of the log file considerably.

Let me know what you think!

mkellerman commented 1 week ago

Previous Logs:

OpenAPS.swift - determineBasal(currentTemp:clock:) - 21 DEV: Start determineBasal
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log: Middleware reason: Nothing changed
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log: Pumphistory is empty!
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log: insulinFactor set to : 55
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log: Using weighted TDD average: 1 U, instead of past 24 h (0 U), weight: 0.65
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log: , Dynamic ratios log: Dynamic Settings disabled
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log: Default Half Basal Target used: 8.9 mmol/L
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log: Autosens ratio: 1;
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log: Threshold set to 3.9
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log: ISF unchanged: 3
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log: CR:10
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log: currenttemp:0 lastTempAge:28652688m, tempModulus:18m
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log: SMB disabled (no enableSMB preferences active or no condition satisfied)
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log: profile.sens:3, sens:3, CSF:5.4
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log: Carb Impact:-4.3mg/dL per 5m; CI Duration:0hours; remaining CI (1.5h peak):0mg/dL per 5m
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log: UAM Impact:-4.3mg/dL per 5m; UAM Duration:0hours
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log: minPredBG: 93 minIOBPredBG: 93 minZTGuardBG: 117
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log:  avgPredBG:117 COB/Carbs:0/0
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log: BG projected to remain above 5.5 for 35minutes
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log: BG projected to remain above 3.9 for 240minutes
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log: naive_eventualBG:,117,bgUndershoot:,-47.5,zeroTempDuration:,240,zeroTempEffect:,216,carbsReq:,-49
JavaScriptWorker.swift - createContext() - 61 DEV: JavaScript log: Setting neutral temp basal of 1U/hr

New Logs:

JavaScriptWorker.swift - aggregateLogs() - 114 DEV: JavaScript log [JSON]: 
{
    "MiddlewareReason": "Nothing changed", 
    "Pumphistory": "empty", 
    "InsulinFactor": "55", 
    "WeightedTDDAverage": "1 U", "WeightedTDDAveragePast24h": "0 U", "Weight": "0.65", 
    "DynamicRatiosLog": "Dynamic Settings disabled", 
    "DefaultHalfBasalTarget": "8.9 mmol/L", 
    "AutosensRatio": "1", 
    "Threshold": "3.9", 
    "ISFUnchanged": "3", 
    "CR": "10", 
    "Currenttemp": "0", "LastTempAge": "28652688m", "TempModulus": "18m", 
    "SMBDisabled": "no enableSMB preferences active or no condition satisfied", 
    "ProfileSens": "3", "Sens": "3", "CSF": "5.4", 
    "CarbImpact": "-4.3mg/dL per 5m", "CIDuration": "0hours", "RemainingCI15hPeak": "0mg/dL per 5m", 
    "UAMImpact": "-4.3mg/dL per 5m", "UAMDuration": "0hours", 
    "MinPredBG": "93", "MinIOBPredBG": "93", "MinZTGuardBG": "117", 
    "AvgPredBG": "117", "COBCarbs": "0/0", 
    "BGProjected": "remain above 5.5 for 35minutes", 
    "BGProjected": "remain above 3.9 for 240minutes", 
    "NaiveEventualBG": "117", "BgUndershoot": "-47.5", "ZeroTempDuration": "240", "ZeroTempEffect": "216", "CarbsReq": "-49", 
    "NeutralTempBasal": "1U/hr", 
}

marionbarker commented 1 week ago

Summary

This patch builds and runs. Probably needs a bit more work.

testing summary:

Before and After patch, logs appended at the end of this message as zip files.

After the rebuild (and not before), see this error message in logs:

2024-06-23T09:12:27-0700 [OpenAPS] JavaScriptWorker.swift - createContext() - 23 - DEV: JavaScript log: Mismatched number of resumes(2) and suspends(1)!
2

with some extra lines associated with that (I think).

Quantity	Before	After	Delta
parser works	yes	yes	-
last timestamp	2024-06-23T09:11:26-0700	2024-06-23T11:10:58-0700	-
lines	240,427	281,042	40,615
characters	2,307,917	2,766,866	458,949
hrs	9.183	11.183	2.00
lines/hr	26k	-	20k
char/hr	251k	-	229k

Configuration

I applied the patch from this PR to the alpha branch (where a number of PR have been accumulating prior to being merged to dev), commit 3ec6176d.

Testing

started with an SE running Trio alpha attached to an rPi DASH
did a deactivate and pair new with old code/log
save the log to trio-pr331-before folder.
rebuild

Let it run for a while

save to log to trio-pr331-after folder

Configure the python parsing program still works:

Success:

Run OmniLoopMessageParser on the 2 log files.

Note that parser breaks logs into individual pods segments when more than one pod is detected, thus the second pod in the log started just before rebuilding to Trio with patch. The output report for that pod is summarized below.

before:

  This pod status:
            Pod active time (hrs) :    0.0
     Insulin delivered by pod (u) :    3.10 (from last 0x1d)

  Report below is for all messages to date for this pod

            First message in log : 2024-06-23 09:09:36
            Last  message in log : 2024-06-23 09:11:01
  Total elapsed time in log (hrs) :    0.0
       Total Bolus Req in log (u) :    3.10

after

  This pod status:
            Pod active time (hrs) :    2.0
     Insulin delivered by pod (u) :    4.40 (from last 0x1d)

  Report below is for all messages to date for this pod

            First message in log : 2024-06-23 09:09:36
            Last  message in log : 2024-06-23 11:09:06
  Total elapsed time in log (hrs) :    2.0
       Total Bolus Req in log (u) :    4.40

Log files:

Log before the rebuld: log-pr-331-before.txt.zip

Log after the rebuild: log-pr-331-after.txt.zip

nightscout / Trio