search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.14k stars 4.13k forks source link

[helm] Specifying user/pass for external minio #35347

Open aqeelat opened 9 months ago

aqeelat commented 9 months ago

Helm Chart Version

0.53.159

What step the error happened?

Other

Revelant information

When deciding to use external minio, we have to add the credentials this way:

minio:
  enabled: false
  auth:
    rootUser: ""
    rootPassword: ""

because this is how they're specified in the secrets.yaml file https://github.com/airbytehq/airbyte-platform/blob/v0.50.50/charts/airbyte/templates/secret.yaml

  STATE_STORAGE_MINIO_ACCESS_KEY: {{ .Values.minio.auth.rootUser | quote }}
  STATE_STORAGE_MINIO_SECRET_ACCESS_KEY: {{ .Values.minio.auth.rootPassword | quote }}

This is unintuitive and undocumented.

I think it should be similar to how externalDatabase is structured.

Relevant log output

No response

marcosmarxm commented 9 months ago

Thanks for reporting the issue @aqeelat at the moment team is refactoring the Helm chart to improve the user experience related to these components.

djpirra commented 2 months ago

Any update on this?

How do you also specify the external Minio address? With external MinIO installation is not supposed to use the s3 storage type? Very confusing from the documentation.