search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.35k stars 4.16k forks source link

[abctl] Ignored minio.storage.volumeClaimValue value, default value 500Mi used #43357

Closed szemek closed 2 weeks ago

szemek commented 3 months ago

What happened?

I installed abctl local install --chart-version 0.399.0 --values values.yaml with values.yaml

minio:
  storage:
    volumeClaimValue: 1000Mi

Value 1000Mi is ignored.

$ kubectl get pvc
NAME                                     STATUS   VOLUME              CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
airbyte-minio-pv-claim-airbyte-minio-0   Bound    airbyte-minio-pv    500Mi      RWO            standard       <unset>                 26m
airbyte-volume-db-airbyte-db-0           Bound    airbyte-volume-db   500Mi      RWO            standard       <unset>                 26m

$ kubectl get pv
NAME                CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                                  STORAGECLASS   VOLUMEATTRIBUTESCLASS   REASON   AGE
airbyte-minio-pv    500Mi      RWO            Retain           Bound    airbyte-abctl/airbyte-minio-pv-claim-airbyte-minio-0   standard       <unset>                          26m
airbyte-volume-db   500Mi      RWO            Retain           Bound    airbyte-abctl/airbyte-volume-db-airbyte-db-0           standard       <unset>                          26m

What did you expect to happen?

abctl should use value from values.yaml when creating PersistentVolume. At this moment default value 500Mi is used.

client.go#L114-L130 client.go#L20

Abctl Version

```console $ abctl version version: v0.11.1 ```

Docker Version

```console $ docker version Client: Version: 26.1.4 API version: 1.45 Go version: go1.21.11 Git commit: 5650f9b Built: Wed Jun 5 11:26:02 2024 OS/Arch: darwin/arm64 Context: desktop-linux Server: Docker Desktop 4.31.0 (153195) Engine: Version: 26.1.4 API version: 1.45 (minimum version 1.24) Go version: go1.21.11 Git commit: de5c9cf Built: Wed Jun 5 11:29:12 2024 OS/Arch: linux/arm64 Experimental: false containerd: Version: 1.6.33 GitCommit: d2d58213f83a351ca8f528a95fbd145f5654e957 runc: Version: 1.1.12 GitCommit: v1.1.12-0-g51d5e94 docker-init: Version: 0.19.0 GitCommit: de40ad0 ```

OS Version

```console # On Mac: $ uname -a Darwin mac.local 23.5.0 Darwin Kernel Version 23.5.0: Wed May 1 20:16:51 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T8103 arm64 ```
marcosmarxm commented 3 months ago

cc @airbytehq/platform-deployments

bgroff commented 2 months ago

@colesnodgrass Is there any reason we could not move the persistent volume size into the default values file in abctl? Would that allow a user supplied values file the ability to override this value?

HumbleBeck commented 1 month ago

Hello, any news here?

bgroff commented 2 weeks ago

You can now set those values in a vaules.yaml override file:

https://github.com/airbytehq/airbyte-platform-internal/blob/master/oss/charts/airbyte/values.yaml#L1760