graphnet-team / graphnet

A Deep learning library for neutrino telescopes
https://graphnet-team.github.io/graphnet/
Apache License 2.0
85 stars 86 forks source link

handling of empty DB when merging #675

Closed Aske-Rosted closed 3 months ago

Aske-Rosted commented 4 months ago

This PR introduces an assertion that checks the automatically pulled tables of the database file. Furthermore in the _merge_databases of the SQLiteWriter this assertation leads to a warning and then the file being skipped rather than an error that aborts the merging of the databases.

While empty databases should not appear to often this handling is in my opinion better since the merging of databases is a step in our conversion process which is not parallelized and therefore extremely slow. we also do not have a way of restarting the merging if it was interrupted mid way through meaning that any progress will be lost.

Since a warning is cast the user can then determine whether they want to start over the merging or continue without the content (if any) of the file causing the issue.