dssg / triage

General Purpose Risk Modeling and Prediction Toolkit for Policy and Social Good Problems
Other
181 stars 61 forks source link

Add an option for not persisting model objects to disk? #916

Open shaycrk opened 1 year ago

shaycrk commented 1 year ago

In the same way that running with save_predictions=False can be useful to avoid filling up database space, it would be useful to add optional behavior to delete model objects (maybe matrices as well?) from disk after they're done being used to save space avoid disk full issues during large, exploratory runs.

The main downside here, of course, is making it more expensive to resume an experiment after a crash mid-run, but it might be a reasonable trade-off.