Add functions to help use OCCweb as a file server, including caching

taldcroft commented 2 years ago

Description

This adds a couple of utility functions to facilitate access to files on OCCweb. Both of these handle authentication implicitly and allow specifying either a full URL or a shortened path.

The shortened path can be either a string or a Path (where the latter makes it easier to chain down directories). The shortened path basically starts at the top of where OCCweb can be viewed as a file server.

This was initially developed within #212 but I decided to pull it out as a separate PR, in part because https://github.com/sot/parse_cm/pull/35 needs this functionality.

Testing

[x] Passes unit tests on MacOS
[x] Functional testing

Functional testing

This got a lot of independent testing during developing of #212.

jeanconn commented 2 years ago

Though actually I just ran the tests and I've got a bunch of warnings on HEAD ska3 flight?

kadi/tests/test_occweb.py: 10 warnings
  /proj/sot/ska3/flight/lib/python3.8/site-packages/astropy/utils/data.py:1587: CacheMissingWarning: Remote data cache could not be accessed due to OSError: Intended kadi cache directory cache is actually a file.
    warn(CacheMissingWarning(msg + e.__class__.__name__ + estr))
kadi/tests/test_occweb.py::test_get_occweb_dir[update-str]
  /proj/sot/ska3/flight/lib/python3.8/site-packages/astropy/utils/data.py:1278: CacheMissingWarning: ('Cache directory cannot be read or created (Intended kadi cache directory cache is actually a file.), providing data in temporary file instead.', '/tmp/astropy-download-27411-kt73m9q6')
    warn(CacheMissingWarning(missing_cache, f_name))

taldcroft commented 2 years ago

Closing this, superceded by #212 which further developed these functions.

sot / kadi