change the User-Agent Header selection

taspinar / twitterscraper

Scrape Twitter for Tweets

MIT License

2.4k stars 581 forks source link

change the User-Agent Header selection #184

Closed RachadAbiChahine closed 5 years ago

RachadAbiChahine commented 5 years ago

Hello, I added fake_useragent lib to replace hand made random user agent

taspinar commented 5 years ago

@RachadAbiChahine In the past twitterscraper was using fake_useragent but this resulted in various problems (look in the issues for fake_useragent).

If you really believe it will have added value, I can merge this PR. However, I suggest that you use ua without caching like suggested. Also, can you remove all of the double and redundant entries in requirements.txt? (just add fake_useragent).

taspinar commented 5 years ago

I'll merge this PR for now and add the above mentioned changes later on.