bmos
commented
8 months ago Implementing this earlier in the project would have saved so much time.
Before it can be merged, there should be some method of initiating a full scan (or better yet scanning for just new files) beyond just deleting the pickles.
Scanning over 170 zip files and re-processing changes every time the server starts wastes a lot of development time and is annoying for users who want to quickly reference metrics/stats. By using Python's data pickling functionality, we can re-load the last results instead of re-scanning on load.