search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.46k stars 3.99k forks source link

Custom images all return 504 when trying to set up local connectors #43893

Open dslockhart opened 1 month ago

dslockhart commented 1 month ago

What happened?

I have moved from using the deprecated run-ab-platform.sh script to the abctl homebrew-installed app. I have managed to set it up and test an out-the-box connector but any docker image, which when I try to add a custom connector using a local docker image, I get a 504 message returned. Is there some place I can see further error logs?

What did you expect to happen?

These images work fine on v0.50.22, both locally and in production. I would like to be able to test them locally using abctl app

Abctl Version

```console $ abctl version version: v0.11.1 ```

Docker Version

```console $ docker version Docker Desktop 4.29.0 (145265) ```

OS Version

```console # On Mac: $ uname -a Darwin GB5ANNM0FH1FQML 23.6.0 Darwin Kernel Version 23.6.0: Fri Jul 5 17:54:20 PDT 2024; root:xnu-10063.141.1~2/RELEASE_X86_64 x86_64 ```
colesnodgrass commented 1 month ago

Currently abctl doesn't support an easy way to view logs (there is an ongoing discussion about how best to support this), but as this tool configures kubernetes in docker, you can use existing kubernetes tools to view the logs.

To view the pods

kubectl --kubeconfig ~/.airbyte/abctl/abctl.kubeconfig -n airbyte-abctl get pods

and then to view the logs of a specific pod

kubectl --kubeconfig ~/.airbyte/abctl/abctl.kubeconfig -n airbyte-abctl logs [POD NAME]
dslockhart commented 1 month ago

Thank you for the help with the logs; I'm not finding anything useful inside. Are there any known issues or fixes for 504 errors when trying to set up connectors from local docker images?

marcosmarxm commented 4 weeks ago

@dslockhart you need to load the images to kind, please run: kind load docker-image airbyte/custom-connector:dev --name airbyte-abctl

dslockhart commented 4 weeks ago

Thank you @marcosmarxm, I spoke too soon. I now get a: An unexpected error occurred. Please report this if the issue persists. (HTTP 500) response when trying to add the new connector

dslockhart commented 4 weeks ago

Could it have something to do with this log: 2024-08-15 14:38:17 ERROR i.a.s.a.ApiHelper(execute):49 - Unexpected Exception java.lang.IllegalStateException: Get Spec job failed. ? Because if I run docker run --rm -i airbyte/source-xxx:latest spec I get a successful output

marcosmarxm commented 4 weeks ago

I need to run some tests @dslockhart

dslockhart commented 4 weeks ago

please let me know if you need more thorough logs

jonseymour commented 1 week ago

I think I may have insight into this as documented here:

https://github.com/airbytehq/airbyte/issues/45156

The problem is that the destination check is failed but because nothing actually logs the failure (I've checked - it isn't logged) and the ingress proxy has already timed out, there is no path from the fault in the destination to the eyeballs of a human who can do something about it.