dlt-hub / dlt

data load tool (dlt) is an open source Python library that makes data loading easy 🛠️
https://dlthub.com/docs
Apache License 2.0
2.08k stars 123 forks source link

RESTClient: extend paginators to handle the HTTP POST method #1359

Open burnash opened 2 months ago

burnash commented 2 months ago

Background

The current implementation of paginators in the RESTClient is designed to manage pagination through the HTTP GET method.

Problem

Certain APIs require the use of the HTTP POST method to retrieve paginated data (typically used when the request involves sending a significant amount of data). For example, Notion's Query a database uses JSON in the request body to filter and paginate data.

Task

rudolfix commented 2 months ago

@burnash should we close it?

burnash commented 1 month ago

@rudolfix no, it's not implemented yet.