search

PelicanPlatform / xrdcl-pelican

A Pelican platform-based plugin for the XrdCl interface
Apache License 2.0
1 stars 3 forks source link

Add support for pelican:// prefixes #3

Closed bbockelm closed 7 months ago

bbockelm commented 7 months ago

This adds support for pelican://-style URLs.

With this, the xrootd client (or the client embedded in XCache) can be pointed at pelican:// style URLs. It will always pull the object from an origin by directly invoking the relevant director API.

Three important implementation notes:

  1. This carefully caches (and periodically updates) the federation metadata about where the director can be found meaning that it isn't re-downloaded for each request.
  2. Pelican 7.4.0 doesn't honor HEAD requests to the director endpoint for origins. This contains a workaround where a GET is issued to the director and, on redirect, we switch to a HEAD instead.
  3. This embeds a JSON header-only library (appropriately licensed) for parsing the federation metadata.
bbockelm commented 7 months ago

@matyasselmeci - going to merge this one. With this, we can set:

pss.origin pelican://osg-htc.org

in the xrootd configuration (provided we have a patch to avoid a known-protocol check; see https://github.com/xrootd/xrootd/issues/2171) and the cache will work with the production director.

matyasselmeci commented 7 months ago

Cool! Can you tag a new version?