search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.4k stars 3.97k forks source link

[helm] Cannot use a S3-compatible storage #36896

Open francois-travais opened 5 months ago

francois-travais commented 5 months ago

Topic

Adding endpoint to the s3 storage

Relevant information

I'm using a cloud provider providing S3-compatible storage and I would like to use it to store Airbyte logs instead of running a Minio instance.

Right now we have several options:

In fact I would only need to be able to provide an optional endpoint to the s3 storage. But then I don't see why there would be two different storage for Minio et S3, I think they could share the implementation.

marcosmarxm commented 4 months ago

What provider are you using?

francois-travais commented 4 months ago

I can use Scaleway, OVH or Clever Cloud

Ae-Miko commented 1 month ago

Would be great to see this addressed, I ran into the same issue very recently.