search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.4k stars 3.97k forks source link

Source Google Analytics Data API: Allow to Specify end date #27986

Open comaraDOTcom opened 1 year ago

comaraDOTcom commented 1 year ago

Connector Name

source-google-analytics-data-api

Connector Version

1.0.0

What step the error happened?

Configuring a new connector

Revelant information

For a custom report that pulls google analytics traffic split by channel the data make not have a key on which to deduplicate data where the state is changing over the first 48 hours. Therefore the strategy we are taking is to use two airbyte setups for GA4.

  1. A full-refresh & overwrite for the 3 day window from today - 2 days to today.
  2. A incremental append for the window from 12 months ago to today - 3 days.

We are hoping this solution will strike the balance of keeping costs low and data accurate. Specifying an end date in the config enables this.

Relevant log output

No response

Contribute

comaraDOTcom commented 1 year ago

I've linked a PR for this issue.

davigs commented 3 months ago

Hi Guys, this feature is very important, I need sync the data past but without end data is impossible.

This feature will is implemented?