search

SeanDaBlack / KelloggBot

Kellogg bad | Union good | Support strike funds
GNU General Public License v3.0
398 stars 81 forks source link

Added multithreading for 4x quicker applying #21

Closed BigweldIndustries closed 2 years ago

BigweldIndustries commented 2 years ago

Hey all! I recently came across from r/antiwork and thought I would give this script a try. Thanks to the awesome work of SeanDaBlack and other contributors, I found it does. However, as the selenium snob I am, I felt it had a bit to be desired. First of all, it is now supported on linux, and requires no webdriver installation. This was done via a pip dependency called webdriver_manager. I then sped up the program to send about three at a time (not too many, chrome windows can be laggy). This was done through the futures dependency, which allows the simple creation of threadpools. Best of luck, comrades!

tommmmyb commented 2 years ago

Nicely done. I did a similar thing with my fork, 4 threads at a time for a few hours - 1500+ applications. Now I'm wishing I'd used futures instead of multiprocessing - looks a lot cleaner. Too bad the reCaptcha ruined my fun

BigweldIndustries commented 2 years ago

Yes, futures is incredible. Makes multithreading so much easier, and isn't that hard to implement. Whenever manually working with threads outside of threadpools my brain just dies.

BigweldIndustries commented 2 years ago

Sorry all, this is behind many commits and I am meaning to rework lots of this. I have created a seperate request for webdriver-manager. I will create one later for multiproccessing after captcha and webdriver-manager are implemented. For now, defer to #27