search

docyx / pc-part-dataset

A dataset of PC parts scraped from PCPartPicker
MIT License
167 stars 41 forks source link

Will be any online api available? #2

Closed WilczekCK closed 11 months ago

WilczekCK commented 2 years ago

As the topic says. I'm currently using the local JSON file, but I think it could be a really good api if it will be updated and available for public.

Thanks

docyx commented 2 years ago

I've definitely thought about it.

Since PCPartPicker is pretty much impossible to scrape now with the rate limits and Recaptcha, we should probably find another site.

Maybe scrape from each manufacturer's site for the most reliable data? I don't know.

If there's enough demand for it, I can make it in a couple weeks.

WilczekCK commented 2 years ago

Keep in touch if you need some help, surely it will be released faster :) Will it be written in NodeJS (or some related framework), I'm pretty fluent with it.

docyx commented 2 years ago

Node. I'll get a basic prototype going and let you know when it's up on GH. Thanks

docyx commented 2 years ago

Update: unfortunately I haven't had the time to get started on this. If you need something sooner than, say, within the next month or two, please let me know. Otherwise it'll get done... at some point.

Sorry!

WilczekCK commented 2 years ago

No problem, I'm using the JSON file right now, so its not necessary, but would be helpful 😁

WalterClementsJr commented 2 years ago

Sadly, it's against PCPartPicker's TOS (in the Access section).

PierrunoYT commented 1 year ago

Can you upload in jsonl format?

docyx commented 12 months ago

@PierrunoYT just uploaded https://github.com/docyx/pc-part-dataset/tree/main/data

docyx commented 11 months ago

No plans for an API. Closing.