airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.47k stars 3.99k forks source link

Source Drift: timing out - ingestion is impossible #21194

Open monik-a opened 1 year ago

monik-a commented 1 year ago

Airbyte version: example is 0.40.22 Deployment: Kubernetes using official charts Source Connector and version: Snapchat 0.1.8 Destination Connector and version: Snowflake 0.4.40 Step where error happened: Sync job

Environment

Current Behavior

The connection fails every day with 500 or 504 errors. There's no issues with the Drift API. In the logs you see also that most of the data gets retrieved, although none of it gets saved since the whole integration fails.

Expected Behavior

Sync should be successfull dc7dd622_3d4a_4164_bbf2_eb4e35990748_logs_682_txt.txt

Steps to Reproduce

  1. Set up Drift connection with the above versions (destination can be arbitrary)
  2. Run the sync
  3. 3 attempts will fail
sherifnada commented 1 year ago

cc @natalyjazzviolin FYI this should go to team/connectors-python, relabelled

jzamanxometry commented 4 months ago

+1