search

Gertje823 / Vinted-Scraper

This is a tool to scrape/download images and data from Vinted & Depop using the API and stores the data in a SQLite database.
GNU General Public License v3.0
91 stars 21 forks source link

Download Whole Website Instead of User Specific Information #1

Closed einfachluki closed 2 years ago

einfachluki commented 2 years ago

Hey! First of all, thank you very much for providing such wonderful tool! I wanted to ask, if it would be possible to scrape not only user specific data (by providing User ID), but rather the whole website itself? Many thanks in advance!

Gertje823 commented 2 years ago

Hi,

Unfortunately this script only scrapes the data from the api and puts it in a database. I am not planning to implement a feature that will scrape the whole page.

What kind of info do you mean? If you need some specific info that is also available through the api I can try to implement it in the script.

einfachluki commented 2 years ago

Oh okay! Actually I would need all the products posted on vinted.com as jpg file, not only the products posted by one or more users

Gertje823 commented 2 years ago

You could do so by modifying the script so it starts with userid=0 and add userid+=1 just before the end of the loop. This way you would loop over all users and download all their posts.

I tried it in the past but it is quite slow and inefficient but it works 😄 I downloaded about 5.400.000 items (300gb) in a couple days.