Allow disabling virtual-hosted-style addressing

[DamienMatias]

Tell us more about this new feature.

Hello,

I was wondering if there was a way to force the usage of path style requests instead of the default virtual-hosted-style addressing ? This is actually possible with mountpoint-s3 as you can see here in their documentation.

To disable virtual-hosted-style addressing, use the --force-path-style command-line flag to instead send requests to https://example.com/docexamplebucket/.

This would allow us using the LakeFS S3 Gateway and potentially other usages that leverage tools still built around the path style addressing.

Thank you 🙏

[fuatbasik]

Hello, @DamienMatias ! Thank you for your interest in Amazon S3 Connector for PyTorch. We'll discuss this internally report back here when we have something concrete to share.

[N-o-Z]

@DamienMatias Hi - lakeFS maintainer here 👋🏽 As a WA for this issue you can configure LAKEFS_GATEWAYS_S3_DOMAIN_NAME to enable lakeFS to work with a virtual host

[devel4848]

I'am also interested in disabling virtual-hosted-style addressing. I would like to use S3 Connector for PyTorch with a S3-compatible Ceph storage that can not be configured for virtual-hosted-style addressing because of the DNS implication. Only path-style addressing can be used in our context.

[jamesbornholt]

The S3 client we use supports disabling virtual-hosted-style addressing here, so I think this would just be a matter of plumbing through a new flag from the various constructors (S3IterableDataset.from_prefix and friends) to the Rust constructor here, similar to #195. I'm not sure we're going to get to this in the short term, but we'd happily review a PR!

[dannycjones]

I'll reopen this as while its merged, there's no new release. (Looks like I mistakenly linked closing the PR to closing this issue.)

This should be supported in the next published version!

[IsaevIlya]

This feature was release in v1.2.5, so closing the request.