SEO python scraper to extract data from major searchengine result pages. Extract data like url, title, snippet, richsnippet and the type from searchresults for given keywords. Detect Ads or make automated screenshots. You can also fetch text content of urls provided in searchresults or by your own. It's usefull for SEO and business related research tasks.
Hi. I have tried to send the requests with some proxies in a proxy.txt file.
But it doesn't work, as many selenium instances open in the browser and everything collapses.
Selenium opens up even if headless mode is True.
Then, I get this error:
scrapcore.scraper.selenium - WARNING - 'NoneType' object has no attribute 'group'
Furthermore, the proxy file appears not to be working if only one http proxy without user:pass is configured.
Hi. I have tried to send the requests with some proxies in a proxy.txt file. But it doesn't work, as many selenium instances open in the browser and everything collapses. Selenium opens up even if headless mode is True. Then, I get this error: scrapcore.scraper.selenium - WARNING - 'NoneType' object has no attribute 'group' Furthermore, the proxy file appears not to be working if only one http proxy without user:pass is configured.