airbytehq / quickstarts

180 stars 45 forks source link

Pagination Handling in Github COnnector #106

Open Atal-Durgesh opened 4 months ago

Atal-Durgesh commented 4 months ago

Hi Team,

I want to write github connector using pyAirbyte. Just wanted to understand how we can handle pagination and how can we pull incremental data using my own cache. I don't find any configuration available to apply pagination and also how we can maintain a checkpoint to pull the last refreshed data from.