search

geniass / ebucks-dealz

https://geniass.github.io/ebucks-dealz/
https://geniass.github.io/ebucks-dealz/
Apache License 2.0
8 stars 7 forks source link

Deal URLs randomly change #1

Open geniass opened 3 years ago

geniass commented 3 years ago

Some products have multiple pages on the store, so the scraper randomly chooses one as the page URL (race condition).

This would make change notifications difficult unless we ignore the URL or something.

tinuva commented 3 years ago

In my own scraper I based this on the product id alone which is the same in all urls for the different categories.