Gertje823 / Vinted-Scraper

This is a tool to scrape/download images and data from Vinted & Depop using the API and stores the data in a SQLite database.
GNU General Public License v3.0
91 stars 21 forks source link

Depop 'products' issue #53

Closed Laokza closed 4 months ago

Laokza commented 5 months ago

Traceback (most recent call last): File "D:\Lol\Vinted-Scraper-main\scraper.py", line 709, in download_depop_data(userids) File "D:\Lol\Vinted-Scraper-main\scraper.py", line 517, in download_depop_data get_all_depop_items(data, baseurl, product_ids, args, begin, s) File "D:\Lol\Vinted-Scraper-main\scraper.py", line 318, in get_all_depop_items for i in data['products']:


KeyError: 'products'
Laokza commented 5 months ago

Traceback (most recent call last): File "D:\Lol\Vinted-Scraper-main\scraper.py", line 559, in download_depop_data Category = product_data['group']


KeyError: 'group'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Lol\Vinted-Scraper-main\scraper.py", line 709, in <module>
    download_depop_data(userids)
  File "D:\Lol\Vinted-Scraper-main\scraper.py", line 561, in download_depop_data
    Category = product_data['categoryId']
               ~~~~~~~~~~~~^^^^^^^^^^^^^^
KeyError: 'categoryId'
Laokza commented 5 months ago

First one when running with -d -g

Second one when running with -d only

Laokza commented 4 months ago

I've done a bit of testing with different sellers and I believe the issue may be with the videos. I'm by no means good at coding and whatnot so idk if that's the true issue but I'm able to download non videos just fine and when it gets to a video it errors. for source in x['sources']:

Gertje823 commented 4 months ago

@Laokza Thanks, I will take a look at it this weekend. Do you have an example account that gives this error?

Laokza commented 4 months ago

Yes, lea_card is a perfect exemple since the first post contains a video. At the time of writing this. I'm able to download the first 3 images of the post but once it gets to the video it errors. Here's the full log: Traceback (most recent call last): File "D:\Lol\Vinted-Scraper-main\scraper.py", line 709, in download_depop_data(userids) File "D:\Lol\Vinted-Scraper-main\scraper.py", line 661, in download_depop_data for source in x['sources']: ~^^^^^^^^^^^ KeyError: 'sources'

Gertje823 commented 4 months ago

Depop changed a field in the json response. I edited the code in de59be3 and it should be working now.

Laokza commented 4 months ago

Tested with new code and I'm getting this error:

Traceback (most recent call last): File "D:\Lol\Vinted-Scraper-main\scraper.py", line 563, in download_depop_data Category = product_data['group']


KeyError: 'group'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Lol\Vinted-Scraper-main\scraper.py", line 713, in <module>
    download_depop_data(userids)
  File "D:\Lol\Vinted-Scraper-main\scraper.py", line 565, in download_depop_data
    Category = product_data['categoryId']
               ~~~~~~~~~~~~^^^^^^^^^^^^^^
KeyError: 'categoryId'

Sorry for posting so many errors :/