Speed up the OTTO scrape - Githubissues

calgo-lab / green-db

The monorepo that powers the GreenDB.

https://calgo-lab.github.io/green-db/

22 stars 2 forks source link

Speed up the OTTO scrape #126

Closed itrajanovska closed 1 year ago

itrajanovska commented 1 year ago

After adding some new changes to the OTTO scraping:

Scraping unsustainable electronics and;
Adding more categories to the electronics list our scraping lasts longer in terms of days (more than 4 days right now).

For that matter we need to address these to comments https://github.com/calgo-lab/green-db/issues/120#issuecomment-1403611196 https://github.com/calgo-lab/green-db/issues/120#issuecomment-1403635898