pathwaycom / pathway

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
https://pathway.com
Other
4.34k stars 139 forks source link

Download files in parallel in gdrive connector #58

Open KamilPiechowiak opened 6 months ago

KamilPiechowiak commented 6 months ago

Is your feature request related to a problem? Please describe. GDrive connector downloads files sequentially (new download starts when a previous finishes). It makes reading from gdrive slow.

Describe the solution you'd like GDrive starting to download multiple files at once.