c3-time-domain / SeeChange

A time-domain data reduction pipeline (e.g., for handling images->lightcurves) for surveys like DECam and LS4
BSD 3-Clause "New" or "Revised" License
0 stars 4 forks source link

Reduce disk usage of tests #299

Open rknop opened 1 month ago

rknop commented 1 month ago

Right now the tests use nearly 7GB in the cache directory by the time they're done.

Go through the tests and make sure that (a) things that don't really need to be cached aren't cached (e.g. the "download from NOIRLab" tests that only download from NOIRLab), and (b) that we aren't using more images than we really need. Right now there are multiple DECam exposures that get downloaded; it should be possible to rewrite some tests to use a different image so that we only need one.

I'm adding the LIMIT_CACHE_USAGE env. var that removes most of the cache usage. That does mean that some things need to get downloaded and processed, which will slow down the tests. It'd be nice to not need to use this on github, but right now we're running out of disk space.