metachris / metachris.com-comments

Comments for metachris.com
https://www.metachris.com
0 stars 0 forks source link

Python Thread Pool #8

Open utterances-bot opened 2 years ago

utterances-bot commented 2 years ago

Python Thread Pool · Chris Hager

A thread pool is a group of pre-instantiated, idle threads which stand ready to be given work. These are often preferred over instantiating new threads for each task when there is a large number of (short) tasks to be done rather than a small number of long ones. Suppose you want do download 1000s of documents from the internet, but only have resources for downloading 50 at a time.

https://www.metachris.com/2016/04/python-threadpool/

Blessvskp commented 2 years ago

I am new to using threads. I used simple threads, one for each task, but could not find reduction in time of execution.

My modules are required to download data from a stock exchange and store the downloaded data in a data frame for analysis. In each run of download, there are 158 download tasks. From web articles, I understand that thread pooling has concurrency feature. This concurrency feature might reduce the time of 158 download tasks.

In your code above, a real life example could have been given implementing your code to let your readers see the benefit of pooling. To be honest, I may not be able to apply your code for real situation, as I could not grasp the full functionality of your code.

As it is not out of place, I would like to mention that geeks that are extremely strong in technicals, miss out to give vitals to their audiences (jump out steps). It makes difficult to understand the subject.

I request you to please give a real life example with your code.

Your response will be appreciated.

Bless