ropensci / rnoaa

R interface to many NOAA data APIs
https://docs.ropensci.org/rnoaa
Other
330 stars 84 forks source link

Option to disable caching #406

Open lpiep opened 2 years ago

lpiep commented 2 years ago

For data retrieval functions such as lcd, isd, etc, it would be helpful to include an option not to cache a cleaned version of the data, rather than having to delete the cached files manually with lcd_cache$delete and similar functions. The isd function has a cleanup option already, although this only removes the raw .gz files.

My use case is that I'm using rnoaa to periodically download and process LCD data, and I want to always be getting the data as it appears on NOAA's servers.

Let me know if this seems like a reasonable feature request. Happy to help with implementation.

sckott commented 2 years ago

have you seen https://docs.ropensci.org/rnoaa/reference/rnoaa_caching.html ?

lpiep commented 2 years ago

Thanks @sckott, I did read through that. I see I can clear the cache when I'm done (e.g. with lcd_cache$delete_all(), but it's not obvious to me how to disable caching entirely. That is, prevent cache files from being stored in the home directory at all.

sckott commented 2 years ago

force=TRUE ?

lpiep commented 2 years ago

For lcd, at least, there isn't a force option documented, and using it doesn't appear to prevent the cache file from being used.

> rnoaa::lcd_cache$delete_all()
> x <- rnoaa::lcd('72027763843', 2022)
> x2 <- rnoaa::lcd('72027763843', 2022, force = TRUE)
using cached file: ~/.cache/R/noaa_lcd/2022_72027763843.csv
date created (size, mb): 2022-02-11 04:59:29 (1.705)

Even on isd, which does have aforce argument, it doesn't stop the rds file from being saved to the cache, it only prevents it from being used.

> rnoaa::isd_cache$delete_all()
> x <- rnoaa::isd('720277', '63843', 2022, force = TRUE)
<path>~/.cache/R/noaa_isd/720277-63843-2022.gz

> list.files(rnoaa::isd_cache$cache_path_get())
[1] "720277-63843-2022.rds"

I suppose that in either case, explicitly clearing the cache after downloading is sufficient for my purposes. I thought though that it would be nice to have finer control over what rnoaa saves into the home directory, especially when the user specifies not to use those files.

sckott commented 2 years ago

@djhocking i'll let you take it from here, just tried to chime in while i had a few min (@lpiep i'm the former maintainer)