scrapfly / python-scrapfly

Scrapfly Python SDK for headless browsers and proxy rotation
https://scrapfly.io/docs/sdk/python
Other
28 stars 8 forks source link

does async_scrape consume less resources than scrape while waiting? #5

Closed louisabraham closed 3 years ago

jjsaunier commented 3 years ago

Yes and no, while your application is calling / waiting I/O (tcp / http calls) it block the CPU and enter in idling phase, doing nothing. Async approach allow your application (without thread / multi process) to process other stuff while waiting, it's cooperative multi tasking (coroutines). So without async approach you are in fact wasting / not using the CPU core at it's full power so yes it's consume less ressources because you are doing less and with async it will consume more ressources but you are doing more stuff at.

Is that answering your question ?