elastic / connectors

Source code for all Elastic connectors, developed by the Search team at Elastic, and home of our Python connector development framework
https://www.elastic.co/guide/en/enterprise-search/master/index.html
Other
69 stars 124 forks source link

[s3] Allow usage of S3-Compatible endpoints #2758

Open nanderson94 opened 3 weeks ago

nanderson94 commented 3 weeks ago

Problem Description

Many cloud providers and self-hosted object storage software utilize S3-compatible APIs which are compatible with boto3 with some extra configuration. Currently, those configuration options are not exposed to the users.

Proposed Solution

Pass-through options commonly utilized by S3-compatible endpoints:

Alternatives

N/A

Additional Context

Rclone's documentation provides a good list of S3-compatible providers: https://rclone.org/s3/

nanderson94 commented 3 weeks ago

Changes have been drafted in my fork: https://github.com/elastic/connectors/commit/379fe6c099f0c01fa8aa3614bd38a00b0e3a18af