Integrate scraped Jakov data

Zivic / GPU-price-tracker

Web scraper and price aggregator built with Next.js 13 and TypeScript

MIT License

2 stars 0 forks source link

Closed Zivic closed 1 year ago

Zivic commented 1 year ago

Figure out any inconsistencies in naming conventions
Try to get as many optional parameters as possible
Handle edge cases
Figure out how to detect whether scraped object already exists if name is slightly different
Create/update DB entries accordingly

Zivic commented 1 year ago

[ ] Handle missing data (memory) by accessing a public GPU api
[ ] Ensure same type is used for both scraper's responses
[ ] Generalize backend "webscraper" function so it can be used for writing any data of this type to the DB
[ ] This also means generalizing the image upload function so it appends the correct source name