search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.34k stars 4.15k forks source link

[source-pinterest] backfill is not working #48056

Open dslockhart opened 1 month ago

dslockhart commented 1 month ago

Connector Name

source-pinterest

Connector Version

2.0.16

What step the error happened?

During the sync

Relevant information

Using the stream 'ad_group_report', I have set the connector start date to 90 days ago (defined as the max backfill window), and yet when I run it I only manage to get data for the current day. I have tried clearing down and refreshing the data, but this has not helped. This is not the case in any of our other connectors.

Relevant log output

No response

Contribute

marcosmarxm commented 3 weeks ago

Do you have only one Pinterest connector and other connectors with backfill?

dslockhart commented 3 weeks ago

We have only one connector for Pinterest, with 4 streams. In any other connector, whether custom made or out-the-box, we can pass the start date/end date and the run will backfill, regardless of state. It looks to me that the state is prohibiting the fetch of any data before the current day, which strikes me as flawed logic as we cannot and should not manually alter the state.