search

bluesky / databroker

Unified API pulling data from multiple sources
https://blueskyproject.io/databroker
BSD 3-Clause "New" or "Revised" License
33 stars 45 forks source link

Support for Object Store #613

Open gwbischof opened 3 years ago

gwbischof commented 3 years ago

It would be nice to be able to lookup files by a uid instead of a file path. This would mean that we wouldn't have to update the mongo documents (resource root + file_path) depending on where we are looking at the data from. It might be nice to have a file search path, which would be a place to search for files by their uid. Also I think object store support would be useful.

stuartcampbell commented 3 years ago

Maybe just having a URI or something would be more flexible for the cases where we don't have or want a posix file path

tacaswell commented 3 years ago

The resource_path, root, and path_semantics keys in the resource documents are all just strings, there is nothing preventing us from deciding to treat them as URIs rather than paths. This is how it is documented in the resource schema:

        "resource_path": {
            "type": "string",
            "description": "Filepath or URI for locating this resource"
        },

If we move to an object store model, we are still going to need to maintain both the root and resource_path analogs (e.g. "what bucket" and "what object in said bucket").