search

PelicanPlatform / pelicanfs

An fsspec implementation that uses the pelican client
https://pelicanplatform.org/
Apache License 2.0
1 stars 4 forks source link

Cache information about the namespace #39

Closed bbockelm closed 6 months ago

bbockelm commented 6 months ago

This PR adds a cache which tracks information about the various namespace prefixes.

On cache miss, the prefix and the list of endpoints are put into the cache (caching for up to 15 minutes). This information is reused on subsequent opens of the object.

Additionally, to ease my local testing, this registers the osdf:// and pelican:// schemes with fsspec, allowing this to be a valid mini-test:

import fsspec
with fsspec.open("osdf:///ospool/uc-shared/public/OSG-Staff/validation/test.txt") as of:
    print(of.read(1024))
bbockelm commented 6 months ago

Just to get things going here, I'm going to merge.

We now have unit tests! 🎉