Open kilobyte2007 opened 4 years ago
Hi kilobyte2007, Sorry for my late reply.
The method of proxycrawl api doesn't work anymore. As a result, I figure out a desperate approach: Using the pywinauto to download the html file first and scrape all the data locally.
It work so far. I have uploaded all the code in this repositories. you can try it
Thank you very much
@wchan757 Hey, I have been trying the steps you mentioned to download the html and then scrape it but I keep running into this error. ElementNotFoundError: {'title_re': '.*Market Share Stats', 'backend': 'uia', 'visible_only': False}
Does this still work of now? I am getting the 520 response code while making the scrape request. The url prefix is right. Is it something related to their anti-bot mechanisms?