airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.07k stars 4.11k forks source link

abctl getting stuck on EC2 #42896

Closed billlyzhaoyh closed 1 month ago

billlyzhaoyh commented 3 months ago

What happened?

abctl local install hangs after following the guide here: https://docs.airbyte.com/using-airbyte/getting-started/oss-quickstart#using-an-ec2-instance-with-abctl

The installation gets stuck here:

  INFO    Using Kubernetes provider:
            Provider: kind
            Kubeconfig: /home/ec2-user/.airbyte/abctl/abctl.kubeconfig
            Context: kind-airbyte-abctl
 SUCCESS  Found Docker installation: version 25.0.3
 SUCCESS  Port 8000 appears to be available
 SUCCESS  Existing cluster 'airbyte-abctl' found
 SUCCESS  Cluster 'airbyte-abctl' validation complete
  INFO    Namespace 'airbyte-abctl' already exists
  INFO    Persistent volume 'airbyte-minio-pv' already exists
  INFO    Persistent volume 'airbyte-volume-db' already exists
  INFO    Persistent volume claim 'airbyte-minio-pv-claim-airbyte-minio-0' already exists
  INFO    Persistent volume claim 'airbyte-volume-db-airbyte-db-0' already exists
  INFO    Starting Helm Chart installation of 'airbyte/airbyte' (version: 0.363.0)

What did you expect to happen?

The installation completes successfully with app being up

Abctl Version

```console $ abctl version version: v0.10.0 ```

Docker Version

```console $ docker version Client: Version: 25.0.3 API version: 1.44 Go version: go1.22.4 Git commit: 4debf41 Built: Tue Jul 2 00:00:00 2024 OS/Arch: linux/amd64 Context: default Server: Engine: Version: 25.0.3 API version: 1.44 (minimum version 1.24) Go version: go1.22.4 Git commit: f417435 Built: Tue Jul 2 00:00:00 2024 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.7.11 GitCommit: 64b8a811b07ba6288238eefc14d898ee0b5b99ba runc: Version: 1.1.11 GitCommit: 4bccb38cc9cf198d52bebf2b3a90cd14e7af8c06 docker-init: Version: 0.19.0 GitCommit: de40ad0 ```

OS Version

```console # On Linux: $ cat /etc/os-release NAME="Amazon Linux" VERSION="2023" ID="amzn" ID_LIKE="fedora" VERSION_ID="2023" PLATFORM_ID="platform:al2023" PRETTY_NAME="Amazon Linux 2023.5.20240722" ANSI_COLOR="0;33" CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2023" HOME_URL="https://aws.amazon.com/linux/amazon-linux-2023/" DOCUMENTATION_URL="https://docs.aws.amazon.com/linux/" SUPPORT_URL="https://aws.amazon.com/premiumsupport/" BUG_REPORT_URL="https://github.com/amazonlinux/amazon-linux-2023" VENDOR_NAME="AWS" VENDOR_URL="https://aws.amazon.com/" SUPPORT_END="2028-03-15" ```
oherik commented 2 months ago

@billlyzhaoyh I have the same issue, were you able to resolve it?

abuchanan-airbyte commented 2 months ago

Try looking at the bootloader logs with

docker exec -it airbyte-abctl-control-plane kubectl -n airbyte-abctl logs airbyte-abctl-airbyte-bootloader

I've seen this happen because, for example, the docker system ran out of disk space.