Closed AlecThomson closed 4 months ago
Hi @maaijke - would you be interest this at all?
Hi Alec, I think something went wrong with my earlier reply, sorr for that. YEs I am interested, especially in the formatter! Local IONEX files is already supported, but if you have a better way of doing that I am happy to have a look at that
Ok cool - I'd be curious to see what is currently supported! I'll open a draft PR so you can inspect the modifications I've been using locally. I'm keen to see what you think :)
Hey @maaijke - thanks for the merge! Just wondering, will you push a new release of RMextract along with this?
Hi @maaijke,
It'd be really useful to be able to handle local (or other exotic server) versions of IONEX data. This is often required on HPCs where internet access is locked down - especially for FTP connections! Thankfully,
urllib
can already handle this with e.g.server=file://path/to/the/data
.Along side this, it'd be handy to be able to pass a formatter for the file naming convention used. This is currently somewhat hard-coded in the
get_urllib_IONEXfile
function. My suggestion would be to have an optionalformatter
argument that is called to appropriately format the URL strings, with a lookup table of known formatters e.g.@tjgalvin and myself have been playing with implementing these changes and along the way we also found that, due to the speed of a local cache, some race conditions can occur. We've put in some checks to catch these.
Would you be open to a PR that brings in these changes? All being well, the current behaviour should be completely preserved.