astropy / astroquery

Functions and classes to access online data resources. Maintainers: @keflavich and @bsipocz and @ceb8
http://astroquery.readthedocs.org/en/latest/
BSD 3-Clause "New" or "Revised" License
702 stars 396 forks source link

SDSS caches files despite cache=False #3121

Open JohannesBuchner opened 4 hours ago

JohannesBuchner commented 4 hours ago

On request by @keflavich, in https://github.com/astropy/astroquery/issues/1239#issuecomment-2424022405 I am report a bug.

I expected `SDSS.get_spectra(..., cache=False)' to not cache on disk.

But it does:

$ rm -rf ~/.astropy/cache/download/url/
$ python3 -c 'from astroquery.sdss import SDSS; SDSS.get_spectra(plate=827, fiberID=23, mjd=52312, cache=False)'
Downloading https://data.sdss.org/sas/dr17/sdss/spectro/redux/26/spectra/0827/spec-0827-52312-0023.fits
|===================================================================================================================================================================| 892k/892k (100.00%)         0s
$ ls ~/.astropy/cache/download/url/
0ad9ac9e9a4231884297d457accb4027

Also setting SDSS.cache_location does not help

$ rm -rf ~/.astropy/cache/download/url/
$ python3 -c 'from astroquery.sdss import SDSS; SDSS.cache_location = "/tmp/"; SDSS.get_spectra(plate=827, fiberID=23, mjd=52312, cache=False)'
Downloading https://data.sdss.org/sas/dr17/sdss/spectro/redux/26/spectra/0827/spec-0827-52312-0023.fits
|===================================================================================================================================================================| 892k/892k (100.00%)         0s
$ ls ~/.astropy/cache/download/url/
0ad9ac9e9a4231884297d457accb4027

version:

$ python3 -c 'import astroquery; print(astroquery.__version__)'
0.4.7
keflavich commented 4 hours ago

Thanks @JohannesBuchner , this clarifies the issue. The cache keyword simply isn't being passed, and on top of that, astropy's default file download caching system is being used rather than astroquery's, so the cached files are being put in the wrong place.

I think I can solve this pretty quickly, first by passing the cache kwarg here: https://github.com/astropy/astroquery/blob/3e26521019d12fb0996135bcffb0f72a64289bec/astroquery/sdss/core.py#L776

and then, possibly, by modifying FileContainer to allow a module-specified path: https://github.com/astropy/astroquery/blob/3e26521019d12fb0996135bcffb0f72a64289bec/astroquery/utils/commons.py#L298