search

docyx / pc-part-dataset

A dataset of PC parts scraped from PCPartPicker
MIT License
172 stars 41 forks source link

Update for newer parts #1

Closed arjanvillon closed 3 years ago

arjanvillon commented 3 years ago

Hi! This is an amazing app. I would just like to suggest to do a new update again for the parts list under the data folder. Thank you very much for this.

docyx commented 3 years ago

Unfortunately PCPP have improved their countermeasures for bots / scrapers, which makes me unable to update the data. I keep getting 503s even with a VPN enabled.

I plan to rewrite in Selenium / maybe TypeScript + Puppeteer to circumvent this. Give me a few days to a week and I'll get this back to where it was (:

data has been updated

arjanvillon commented 3 years ago

Thank you very much. Can't wait for this new app. I'll follow you so I can see if I can help someday on your work :)

docyx commented 3 years ago

@arjanvillon @Ricebowl23 @burakdmir Apologies for the delay - data has been updated (: