search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.3k stars 4.15k forks source link

[source-facebook-marketing] Ad Creatives fails with "Please reduce the amount of data you're asking for" #38025

Open ChloeBellm opened 6 months ago

ChloeBellm commented 6 months ago

Connector Name

source-facebook-marketing

Connector Version

v2.1.7

What step the error happened?

During the sync

Relevant information

Source - Facebook Marketing fails with Please reduce the amount of data you're asking for, then retry your request on adcreatives stream.

This error also happens on the API so potentially need to use an alternative endpoint for the ad creatives data.

We could instead use /ads?fields=creative{id,...}, i.e. pull in the creative fields using the ads endpoint.

https://developers.facebook.com/docs/marketing-api/reference/ad-creative

Relevant log output

Status:  500
  Response:
    {
      "error": {
        "code": 1,
        "message": "Please reduce the amount of data you're asking for, then retry your request"
      }
    }

Contribute

marcosmarxm commented 6 months ago

Hello @ChloeBellm, thank you for pointing out this issue. Could you tell us the amount of data you're to access? Also, do you get the same error when using the API request manually? I've asked the connector team to investigate this further in future sprints.

ChloeBellm commented 6 months ago

Thanks @marcosmarxm for the quick response!

Could you tell us the amount of data you're to access?

We get this error when trying to request a small amount of data, e.g. with adcreatives?limit=1&fields=id. We think it might relate to the 50k limit - the account has more than 50k creatives.

Also, do you get the same error when using the API request manually? I've asked the connector team to investigate this further in future sprints.

We do get the same error directly using the API.

The reason for suggestion of using /ads with the creative field is that way we can just get relevant creatives and ads has a date field we could use to fetch only the latest records.