Gertje823 / Vinted-Scraper

This is a tool to scrape/download images and data from Vinted & Depop using the API and stores the data in a SQLite database.
GNU General Public License v3.0
91 stars 21 forks source link

Vinted scraper stops early #30

Closed andbilp closed 1 year ago

andbilp commented 1 year ago

When scraping a profile on vinted with many listings, it doesn't seem to scrape the pictures of every item.

I tried scraping a profile with 192 items listed and it only scraped 356 pictures, which is between 94-98 items I believe. I think it stopped after scraping the first page, which is 96 items, and doesn't go to the second page.

Screenshot_1

Gertje823 commented 1 year ago

Hmm, that's weird.

Just tried it with an random account that has around 250 items and it downloaded 593 images. I manually selected 10 random item images from the Vinted profile and checked if they were downloaded and they were.

So, I could not reproduce the issue. Are you seeing this issue only by 1 specific account or multiple accounts? And do you have an example profile so I can test this?

andbilp commented 1 year ago

I've only tried it with one big account tbh, here's the user ID: 25894630

From what I can tell it downloads most/all of the items that are on the top of the page, but not the ones towards the bottom of the page.

For reference here are some items that it didn't download for this user: https://www.vinted.be/heren/kleding/truien-en-capuchontruien/truien-met-ronde-hals/354086576-zwarte-sweater https://www.vinted.be/dames/kleding/jassen-en-jacks/jackets/bomberjacks/453290679-bomberjacket https://www.vinted.be/dames/kleding/topjes-en-shirts/blouses/453296015-bloesje-zara https://www.vinted.be/heren/kleding/truien-en-capuchontruien/overige/453301339-sweater-hm https://www.vinted.be/dames/kleding/hoodies-en-sweatshirts/overige/453313108-pull-hm

Gertje823 commented 1 year ago

You were right. I fixed the issue in https://github.com/Gertje823/Vinted-Scraper/commit/2c63cc46b11e56dfc5fe1d96dcef00a961f2ffc2

It should now download all the items :)