Iamcerba
commented
1 year ago Answering my question:
I've found in the code call to loop.run_in_executor, thus each catalog entry is synchronized in a separate Thread and inside these threads "asyncio" isn't being used.
Do you have real control over concurrency?
From what I see in the code there is "max_workers" parameter in config that is used to create custom ThreadPoolExecutor. What I know from the documentation is that loop.set_default_executor works together with loop.run_in_executor and I'm not sure that it works with asyncio.gather. Thus all catalog entries are synced in parallel in a single thread (using async io event loop) without any limit in concurrency.
Correct me if I'm wrong.