As it stands, a year file will never contain data for december and a month file will never contain data for the last day.
Scenario: we receive entries every minute. The last entry received was 2024-12-31T59:59:59Z. It was written to the day-file (ns-day/2024-12-31.json). Even assuming something triggered a dirty write, the last entry in ns-month/2024-12.json was for 2024-12-30T23:59:59Z (because data for the 31st went into the day-file), and the last data in ns-year/2024.json was for 2024-11-31T23:59:59Z (because data for december went in the month-file). Before the day rollover, we didn't want december data in the year file, and we didn't want the 31st's data in the month file.
The year-file flush is an absolute requirement to make "load the previous year's data" work (so we have some historical data if we rebooted on january 1st), but the month flush is only really an issue for backup purposes. In the "normal" single-entry upload scenario, there will be no need to flush the day file, but may be sensible for symmetry's sake.
Alternatively, we could treat these as "archive" files and store them separately?
Hmm. I'm now rethinking the purpose of the day/month/year split. What is the justification for the month files?
Decision: Month files are for import to nightscout. They should be POST-able to nightscout-js (or nightscout-go). They can also be used for any interested parties to use for low-memory analysis. Year files have the same purpose, but for things that can handle more data in one go :)
As it stands, a year file will never contain data for december and a month file will never contain data for the last day.
Scenario: we receive entries every minute. The last entry received was 2024-12-31T59:59:59Z. It was written to the day-file (
ns-day/2024-12-31.json). Even assuming something triggered a dirty write, the last entry inns-month/2024-12.jsonwas for 2024-12-30T23:59:59Z (because data for the 31st went into the day-file), and the last data inns-year/2024.jsonwas for 2024-11-31T23:59:59Z (because data for december went in the month-file). Before the day rollover, we didn't want december data in the year file, and we didn't want the 31st's data in the month file.The year-file flush is an absolute requirement to make "load the previous year's data" work (so we have some historical data if we rebooted on january 1st), but the month flush is only really an issue for backup purposes. In the "normal" single-entry upload scenario, there will be no need to flush the day file, but may be sensible for symmetry's sake.
Alternatively, we could treat these as "archive" files and store them separately?
Hmm. I'm now rethinking the purpose of the day/month/year split. What is the justification for the month files?
Decision: Month files are for import to nightscout. They should be POST-able to nightscout-js (or nightscout-go). They can also be used for any interested parties to use for low-memory analysis. Year files have the same purpose, but for things that can handle more data in one go :)