search

PelicanPlatform / xrootd-s3-http

An XRootD plugin that allows Pelican to interface with s3/http server backends
Apache License 2.0
2 stars 6 forks source link

Add ability to configure "path-style" hosting #10

Closed jhiemstrawisc closed 9 months ago

jhiemstrawisc commented 9 months ago

Right now we very much assume the S3 endpoint is configured for virtual hosting of buckets. This is because Amazon has said that path-style URL access is being deprecated. However, in being careful to distinguish between S3 the protocol and S3 the service from Amazon, we should also support the path-style URL access for any S3 (protocol) endpoints that don't support virtual-hosted requests.

This amounts to adding an XRootD option (maybe s3.access_option = [virtual/path]?) and changing the way we set up URL requests based on the value selected. It might also require changing the way we parse information into the S3 canonicalURI variable, although I haven't investigated that yet.

bbockelm commented 9 months ago

Just some window dressing:

Just to do some bikeshedding, how about:

s3.lookup_style = [virtual / path]

Semantics:

This allows us to, later, add an auto option (MinIO has some hints on how to auto-detect which style is used) and relax the requirement that it is set.