search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.52k stars 4k forks source link

Issue with Facebook Marketing Source Connection Causing Server Crash #39318

Closed ommax-m closed 3 months ago

ommax-m commented 3 months ago

Connector Name

facebook marketing

Connector Version

3.1.0

What step the error happened?

Configuring a new connector

Relevant information

Description:

I'm experiencing a critical issue with connecting my Facebook Marketing source (v3.1.0) to my Redshift destination (latest) on Airbyte version 0.59.1. The problem causes the entire Airbyte server to crash and become inaccessible. Below are the details of the issue:

Environment:

Airbyte Version: 0.59.1 OS/Platform: Kubernetes AWS EKS Instance Type: 2x t2.xlarge Memory / Disk: Adequate based on instance type Source and Destination: Source: Facebook Marketing (v3.1.0) Destination: Redshift (latest)

Steps to Reproduce:

Set up a connection with Facebook Marketing as the source and Redshift as the destination. Test the connection for both the source and destination individually - both tests are successful. Attempt to connect the source to the destination. During the schema fetching step, the process fails with the error "Failed to fetch." The entire Airbyte server crashes and becomes inaccessible for a while.

Issue Details:

The issue consistently occurs at the schema fetching step. The error message displayed is "Failed to fetch." This problem has also been previously observed with Bing Ads, but it is now recurring with Facebook Marketing. There are no additional logs or detailed error messages available through standard methods. I’ve checked the logs using kubectl logs and kubectl describe , but found nothing useful.

Expected Behavior:

The schema should be fetched successfully without crashing the Airbyte server. The connection between the source and destination should be established without issues.

Actual Behavior:

The process crashes at the schema fetching step. The Airbyte server becomes inaccessible for a while after the crash.

Logs:

There are no specific logs indicating the cause of the crash found using:

kubectl logs kubectl describe Additional Information: Similar issues were experienced with Bing Ads previously but were resolved. Seeking guidance on resolving this issue and any suggestions for additional log locations or diagnostics.

Request:

Any help in diagnosing and resolving this issue would be greatly appreciated. If there are any additional steps or logs I should check, please let me know. Thank you for your assistance!

Relevant log output

No response

Contribute

marcosmarxm commented 3 months ago

Do you have only one connection or more? Are you able to monitor the server resources (cpu, memory, disk?)

marcosmarxm commented 3 months ago

Closed due lack of response