Open ubdbra001 opened 2 months ago
Code for splitting data into hourly chunks found in save_emission_rates_as_txt()
within io_py.py
Maybe saving should be incremental, but into a single file for each day? Perhaps saving every hour, or half hour, but also saving if it hits an error.
To-do:
process_pair()
Filename structure:
[date]_emissionrates[flowtype].csv
Just to note that this links directly to #46, so once data are written iteratively I think both issues can be closed (just trying to declutter so that the number of issues isn't totally overwhelming).
Currently the output data files are split into hourly chunks. This should be reassessed, maybe files should represent days.
If changing this then it would also likely be beneficial to increase how often the data is written to a file (so if there are issues the data already processed is not lost). Either write as the data is produced (but this may be too much I/O) or write hourly chunks to a single file per day?