airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.88k stars 4.07k forks source link

abctl unable to install helm #45105

Closed erpadmin closed 1 month ago

erpadmin commented 1 month ago

What happened?

$ abctl local install --host hostname.local INFO Using Kubernetes provider: Provider: kind Kubeconfig: /opt/airbyte/.airbyte/abctl/abctl.kubeconfig Context: kind-airbyte-abctl SUCCESS Found Docker installation: version 27.2.0
SUCCESS Port 8000 appears to be available
INFO No existing cluster found, cluster 'airbyte-abctl' will be created
SUCCESS Cluster 'airbyte-abctl' created
INFO Namespace 'airbyte-abctl' created
INFO Persistent volume 'airbyte-minio-pv' created
INFO Persistent volume 'airbyte-volume-db' created
INFO Persistent volume claim 'airbyte-minio-pv-claim-airbyte-minio-0' created
INFO Persistent volume claim 'airbyte-volume-db-airbyte-db-0' created
INFO Starting Helm Chart installation of 'airbyte/airbyte' (version: 0.524.0)
ERROR Failed to install airbyte/airbyte Helm Chart
ERROR Unable to install Airbyte locally
ERROR unable to install airbyte chart: unable to install helm: rpc error: code = Unavailable desc = error reading from server: EOF

What did you expect to happen?

successful installation

Abctl Version

```console $ abctl version version: v0.13.1 ```

Docker Version

```console $ docker version Client: Docker Engine - Community Version: 27.2.0 API version: 1.47 Go version: go1.21.13 Git commit: 3ab4256 Built: Tue Aug 27 14:17:20 2024 OS/Arch: linux/amd64 Context: default Server: Docker Engine - Community Engine: Version: 27.2.0 API version: 1.47 (minimum version 1.24) Go version: go1.21.13 Git commit: 3ab5c7d Built: Tue Aug 27 14:16:16 2024 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.7.21 GitCommit: 472731909fa34bd7bc9c087e4c27943f9835f111 runc: Version: 1.1.13 GitCommit: v1.1.13-0-g58aa920 docker-init: Version: 0.19.0 GitCommit: de40ad0 ```

OS Version

```console # On Linux: $ cat /etc/os-release NAME="Red Hat Enterprise Linux" VERSION="8.10 (Ootpa)" ID="rhel" ID_LIKE="fedora" VERSION_ID="8.10" PLATFORM_ID="platform:el8" PRETTY_NAME="Red Hat Enterprise Linux 8.10 (Ootpa)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" HOME_URL="https://www.redhat.com/" DOCUMENTATION_URL="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8" BUG_REPORT_URL="https://issues.redhat.com/" REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" REDHAT_BUGZILLA_PRODUCT_VERSION=8.10 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="8.10" ```
erpadmin commented 1 month ago

docker installation notes

$ yum remove docker \
                  docker-client \
                  docker-client-latest \
                  docker-common \
                  docker-latest \
                  docker-latest-logrotate \
                  docker-logrotate \
                  docker-engine \
                  podman \
                  runc
$ yum install -y yum-utils
$ yum-config-manager --add-repo https://download.docker.com/linux/rhel/docker-ce.repo
$ yum install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
$ systemctl enable --now docker
$ useradd -m -d /opt/airbyte airbyte -g docker
$ curl -LsfS https://get.airbyte.com | bash -
erpadmin commented 1 month ago

afdebbug.txt

erpadmin commented 1 month ago

Some lessons are destined to be relearned.

The issue was free space. /var/lib/docker needs to be 10G minimum to launch the app, but it probably should be sized to 50G+, given how docker chews up space in general.

One side note is once the pods start failing to launch, they eventually go into an image pull backoff state and never attempt to restart unless you manually delete the pod.