search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.63k stars 4.02k forks source link

[source-mssql] Large schema fail #31695

Open i2p0 opened 11 months ago

i2p0 commented 11 months ago

Connector Name

source-mssql

Connector Version

2.0.1

What step the error happened?

During the sync

Relevant information

i have a source setup with MS SQL connector that point to a server on premise. the source is set without error, but when i try to create a connection to the destination it fails fetching the schema. the source has a schema with more than 10.000 tables.

it gives the error: Server temporarily unavailable (http.502.oH4FeM1ArpWteRBeQ7PDP2)

Relevant log output

No response

Contribute

CamiloAtencio commented 9 months ago

I am also unable to set up a proper connection from mssql to snowflake

labarreatos commented 8 months ago

I'm also unable to set up a proper connection, but for S3, Clickhouse and postgresql Sources and Destination. When I'm testing the set up for each of the previous one, I obtain the error: Server temporarily unavailable (http.502.r4LwAPhUeLD5aSPCTuXXMz). I don't see any error logs in the worker pod. I've deployed Airbyte in Kubernetes with Helm chart version="0.50.14" and App version="0.50.36". I increased the capacity of the pod (RAM up to 4Gi and CPU up to 4 for the worker pod and up to 1G in Ram & 1 CPU for temporal and webapp pods) but it does not improve anything.

amukhsimov commented 5 months ago

Hello! Any updates? Have the same issue with Oracle DB, large amount of tables, schema fetching fails

evantahler commented 4 months ago

re-assigning to the move team