Cannot download files due to keyerror

goldenagent100 commented 4 years ago

I tried to download the same 2 users in the provided config to test the program, but it failed.

I have included the error and stack trace.

C:\Users\User1\AppData\Local\Programs\Python\Python38\python.exe D:/Documents/code/deviantart-scraper/main.py artwork

there are 2 users

download artworks for user GUWEIZ

Traceback (most recent call last):
  File "D:/Documents/code/deviantart-scraper/main.py", line 54, in <module>
    main()
  File "D:/Documents/code/deviantart-scraper/main.py", line 40, in main
    download_users(api, config, args.option)
  File "D:/Documents/code/deviantart-scraper/main.py", line 11, in download_users
    result = api.save_users_artworks(config.users, config.save_dir)
  File "D:\Documents\code\deviantart-scraper\lib\deviantart.py", line 204, in save_users_artworks
    files = self.save_user_artworks(user, dir_path)
  File "D:\Documents\code\deviantart-scraper\lib\deviantart.py", line 180, in save_user_artworks
    files = pool.map(partial(self.save_artwork, dir_path), artworks)
  File "C:\Users\User1\AppData\Local\Programs\Python\Python38\lib\multiprocessing\pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "C:\Users\User1\AppData\Local\Programs\Python\Python38\lib\multiprocessing\pool.py", line 768, in get
    raise self._value
  File "C:\Users\User1\AppData\Local\Programs\Python\Python38\lib\multiprocessing\pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "C:\Users\User1\AppData\Local\Programs\Python\Python38\lib\multiprocessing\pool.py", line 48, in mapstar
    return list(map(*args))
  File "D:\Documents\code\deviantart-scraper\lib\deviantart.py", line 154, in save_artwork
    download_url = self._download_url(artwork)
  File "D:\Documents\code\deviantart-scraper\lib\deviantart.py", line 111, in _download_url
    url = next(a['src'] for a in artwork['files'] if a['type']=='fullview')
KeyError: 'files'

python version: Python 3.8.1
windows version: Windows 10 Home

MeKHell commented 4 years ago

Hey, I get exactly the same error on an other user (Ellysiumn).

Alex1331xela commented 4 years ago

Same issue here

wynstep commented 4 years ago

Hey, I get exactly the same error on an other user (Ellysiumn).

Same issue here, with exactly the same user. Any suggestion?

YichSouls commented 4 years ago

I think the site has changed the layout thus the error. Can only wait for the update

mitar commented 2 years ago

I made this MR: https://github.com/Kent-Lee/deviantart-scraper/pull/12 but I think it is best to simply use https://github.com/mikf/gallery-dl, which seems to work.

goldenagent100 commented 2 years ago

Thank for the recommendation @mitar, gallery-dl works for many galleries

kent-lee / deviantart-scraper

Cannot download files due to keyerror #8