Open bezkos opened 7 years ago
I did a small tweak so we can have custom folder for local cache DB.
` class RandomUserAgentMiddleware(object): def init(self, crawler): super(RandomUserAgentMiddleware, self).init()
loc = crawler.settings.get('LOCATION') if loc is None: self.ua = UserAgent() else: self.ua = UserAgent(path=crawler.settings.get('LOCATION')) self.per_proxy = crawler.settings.get('RANDOM_UA_PER_PROXY', False) self.proxy2ua = {}`
In settings file, u can have an option like: LOCATION = '/home/user/fake_useragent%s.json' % fake_useragent.VERSION
@bezkos good idea, thanks!
I did a small tweak so we can have custom folder for local cache DB.
` class RandomUserAgentMiddleware(object): def init(self, crawler): super(RandomUserAgentMiddleware, self).init()
In settings file, u can have an option like: LOCATION = '/home/user/fake_useragent%s.json' % fake_useragent.VERSION