The current approach to handling errors and problems with inputted files is not a particularly careful one: simply catch whatever exceptions are raised and display them to the user, not distinguishing between expected exceptions caused by bad input and unexpected exceptions caused by bugs in the code.
In addition, some problems are currently not checked for, as far as I can tell:
Reading in a telemetry file for a different device than the one that is currently being monitored.
Reading in a telemetry file with an inappropriately sized gap in time from the previous telemetry file.
Duplicate timestamps in telemetry data.
Some parts of the device configuration are stored as JSON in the database, and only checked upon retrieval, not when first reading the config file.
Probably a variety of other things -- we were more focused on the happy path when it came to file reading.
I wrote astra/data/dict_parsing.py in the hopes that it could be helpful for parsing YAML/JSON data in a way that could give informative error information -- hopefully it can be of use when addressing some of this stuff.
For integration with the Problems tab: it's probably a good idea to switch from a "new data xor exception" model to a "zero or more pieces of new data and zero or more new errors" model. Some problems likely prevent further processing of data (for example, an unsupported file format), while others probably don't affect data processing at all and simply require raising warnings of other kinds (for example, if the gaps between timestamps suggest that there is a missing telemetry file, all the data from the current telemetry file can still be read in just fine).
The current approach to handling errors and problems with inputted files is not a particularly careful one: simply catch whatever exceptions are raised and display them to the user, not distinguishing between expected exceptions caused by bad input and unexpected exceptions caused by bugs in the code.
In addition, some problems are currently not checked for, as far as I can tell:
I wrote
astra/data/dict_parsing.pyin the hopes that it could be helpful for parsing YAML/JSON data in a way that could give informative error information -- hopefully it can be of use when addressing some of this stuff.For integration with the Problems tab: it's probably a good idea to switch from a "new data xor exception" model to a "zero or more pieces of new data and zero or more new errors" model. Some problems likely prevent further processing of data (for example, an unsupported file format), while others probably don't affect data processing at all and simply require raising warnings of other kinds (for example, if the gaps between timestamps suggest that there is a missing telemetry file, all the data from the current telemetry file can still be read in just fine).