search

taspinar / twitterscraper

Scrape Twitter for Tweets
MIT License
2.39k stars 581 forks source link

A serial mode without using Pool #280

Open axnsantana opened 4 years ago

axnsantana commented 4 years ago

Including an option to run the scrapper in a serial mode without use Pool. The use of Pool is not available in environments like AWS Lambda (#114).

The mode selection uses the previous idea that if poolsize greater than 1, then run in parallel.

I am currently using this in a serverless application in AWS Lambda. The lambda function can be invoked multiple times to mitigate the loss of performance, simulating a parallel execution.

lukaspistelak commented 4 years ago

i have similar issue when run pool inside standard Thread...