The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
Currently CDK lacks inbuilt async capability. Rather than build this ourselves with asyncio and requests, we could use the popular HTTPX library.
HTTPX is inspired by the Pythonic requests package, but is built on a more modern foundation.
Crucially, HTTPX provides native asynchronous APIs.
Describe the solution you’d like
A (spike?) investigation into the use of HTTPX to level-up our async requests capability in the CDK.
Describe the alternative you’ve considered or used
Ousret
commented
9 months ago
Tell us about the problem you're trying to solve
Currently CDK lacks inbuilt async capability. Rather than build this ourselves with
asyncioandrequests, we could use the popular HTTPX library. HTTPX is inspired by the Pythonic requests package, but is built on a more modern foundation. Crucially, HTTPX provides native asynchronous APIs.Describe the solution you’d like
A (spike?) investigation into the use of HTTPX to level-up our async requests capability in the CDK.
Describe the alternative you’ve considered or used
Writing this ourselves