search

alecxe / scrapy-fake-useragent

Random User-Agent middleware based on fake-useragent
MIT License
687 stars 98 forks source link

User agent pool website is down #21

Closed alecsci closed 6 years ago

alecsci commented 6 years ago

Hi,

This is my first time I'm opening an issue to someone else's repository. I see this was not asked / proposed. I am using your library for my student project.

But it looks like the user agents used for rotation from this website http://useragentstring.com/ cannot be generated because the website is not responding.

Can you add some logic to use other websites, or an internal cache in case those websites are down ?

alecsci commented 6 years ago

I see you already provided a variable for this issue.

Sorry.

Closing this issue.