[Deviantart] An unexpected error occurred: KeyError - 'deviation'.

[sophi228]

Hii! I'm using gallery-dl for over a year now, but since the 1.27.5 update, I'm having trouble downloading some literature labeled texts/fanfics from Deviantart. I hope I did nothing wrong. Here are the debugging information:

[gallery-dl][debug] Version 1.27.5 [gallery-dl][debug] Python 3.10.4 - Windows-10-10.0.22631-SP0 [gallery-dl][debug] requests 2.31.0 - urllib3 2.0.7 [gallery-dl][debug] Configuration Files ['%APPDATA%\gallery-dl\config.json'] [gallery-dl][debug] Starting DownloadJob for 'https://www.deviantart.com/starvinglunatic/art/Against-the-world-chapter-1-50968347' [deviantart][debug] Using DeviantartDeviationExtractor for 'https://www.deviantart.com/starvinglunatic/art/Against-the-world-chapter-1-50968347' [deviantart][debug] Using custom API credentials (client-id 29899) [deviantart][info] Requesting public access token [urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.deviantart.com:443 [urllib3.connectionpool][debug] https://www.deviantart.com:443 "POST /oauth2/token HTTP/1.1" 200 126 [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/user/profile/starvinglunatic HTTP/1.1" 200 390 [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /StarvingLunatic/art/50968347 HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/deviation/016057D5-33D2-EC3D-D242-0E9DCF39368A HTTP/1.1" 200 818 [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/deviation/metadata?deviationids%5B0%5D=016057D5-33D2-EC3D-D242-0E9DCF39368A&mature_content=true HTTP/1.1" 200 936 [deviantart][info] Collecting folder information for 'StarvingLunatic' [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/folders?username=StarvingLunatic&offset=0&limit=50&mature_content=true HTTP/1.1" 200 5920 [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/50619809-A51F-09B2-C01C-EAD5CCA7D6BE?username=StarvingLunatic&offset=0&limit=24&mature_content=true&mode=newest HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/50619809-A51F-09B2-C01C-EAD5CCA7D6BE?username=StarvingLunatic&offset=24&limit=24&mature_content=true&mode=newest HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/50619809-A51F-09B2-C01C-EAD5CCA7D6BE?username=StarvingLunatic&offset=48&limit=24&mature_content=true&mode=newest HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/50619809-A51F-09B2-C01C-EAD5CCA7D6BE?username=StarvingLunatic&offset=72&limit=24&mature_content=true&mode=newest HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/50619809-A51F-09B2-C01C-EAD5CCA7D6BE?username=StarvingLunatic&offset=96&limit=24&mature_content=true&mode=newest HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/F3EF72FC-51E3-8CEB-083B-258D6C414FC6?username=StarvingLunatic&offset=0&limit=24&mature_content=true&mode=newest HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/F3EF72FC-51E3-8CEB-083B-258D6C414FC6?username=StarvingLunatic&offset=24&limit=24&mature_content=true&mode=newest HTTP/1.1" 200 3035 [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/F097CA37-DD8F-C43D-C2E0-72A3818309D6?username=StarvingLunatic&offset=0&limit=24&mature_content=true&mode=newest HTTP/1.1" 200 2929 [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/425DD360-D28F-F5A1-76CA-C12D689B5334?username=StarvingLunatic&offset=0&limit=24&mature_content=true&mode=newest HTTP/1.1" 200 4664 [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/7D26849B-B1F3-11F6-037B-94C1E401517B?username=StarvingLunatic&offset=0&limit=24&mature_content=true&mode=newest HTTP/1.1" 200 5006 [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/8E72B18C-CE61-9EA9-121F-9DB5ED29895B?username=StarvingLunatic&offset=0&limit=24&mature_content=true&mode=newest HTTP/1.1" 200 1708 [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/E9F0DC3B-6A44-AA53-9627-5AC5679D082E?username=StarvingLunatic&offset=0&limit=24&mature_content=true&mode=newest HTTP/1.1" 200 4028 [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/60246A9F-71FD-8EDC-F48B-B203A2164613?username=StarvingLunatic&offset=0&limit=24&mature_content=true&mode=newest HTTP/1.1" 200 None [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/61AE6FFD-72C5-BD25-2E14-2F3E09A15DBC?username=StarvingLunatic&offset=0&limit=24&mature_content=true&mode=newest HTTP/1.1" 200 4591 [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/F2BCC72F-2341-3114-F2C2-E13EC2669938?username=StarvingLunatic&offset=0&limit=24&mature_content=true&mode=newest HTTP/1.1" 200 3598 [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/A4BD78D5-3C92-B73C-E5DA-94086806DFAC?username=StarvingLunatic&offset=0&limit=24&mature_content=true&mode=newest HTTP/1.1" 200 3529 [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/comments/deviation/016057D5-33D2-EC3D-D242-0E9DCF39368A?maxdepth=5&offset=0&limit=50&mature_content=true HTTP/1.1" 200 1340 [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET / HTTP/1.1" 200 None [deviantart][debug] Sleeping 2.00 seconds (request) [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /_puppy/dadeviation/init?deviationid=50968347&username=StarvingLunatic&type=journal&include_session=false&expand=deviation.related&da_minor_version=20230710&csrf_token=NS3h3x8dLZW5ODct.sklnfk.JUdnAfMtfmv5xVaWEcoMG9wxPvNmDb0aD4NZVInzMBA HTTP/1.1" 400 118 [deviantart][error] An unexpected error occurred: KeyError - 'deviation'. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues . [deviantart][debug] Traceback (most recent call last): File "C:\Users\klaiu\AppData\Local\Programs\Python\Python310\lib\site-packages\gallery_dl\job.py", line 151, in run for msg in extractor: File "C:\Users\klaiu\AppData\Local\Programs\Python\Python310\lib\site-packages\gallery_dl\extractor\deviantart.py", line 185, in items content = self.eclipse_api.deviation_extended_fetch( KeyError: 'deviation'

[BlueSpiders99]

I am having a similar error with DeviantArt; my KeyError is 'category'.

[gallery-dl][debug] Version 1.25.8 - Executable
[gallery-dl][debug] Python 3.8.10 - Windows-10-10.0.19045
[gallery-dl][debug] requests 2.31.0 - urllib3 1.26.16
[gallery-dl][debug] Configuration Files ['.\\config.json']
[gallery-dl][debug] Starting DownloadJob for 'https://www.deviantart.com/REDACTED/favourites/96088431/new'
[deviantart][debug] Using DeviantartCollectionExtractor for 'https://www.deviantart.com/REDACTED/favourites/96088431/new'
[deviantart][debug] Using custom API credentials (client-id 26320)
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.deviantart.com:443
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/user/profile/REDACTED HTTP/1.1" 200 330
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/collections/folders?username=REDACTED&offset=0&limit=50&mature_content=true HTTP/1.1" 200 None
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/collections/A5E0B283-9E2A-A78F-6A13-4CC68A5E19A0?username=REDACTED&offset=0&limit=24&mature_content=true HTTP/1.1" 200 None
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/deviation/metadata?deviationids%5B0%5D=654092F0-958C-5AD6-84A3-149BBF2D947F&deviationids%5B1%5D=89E8CD81-D7BC-36C2-111E-A2005C93
4BC0&deviationids%5B2%5D=E52BFD20-C24D-538C-9995-5ABB5B52EF94&deviationids%5B3%5D=48D5AD22-5AEB-7BAF-6797-A98DB07BD83B&deviationids%5B4%5D=D421C09E-AF16-30F6-6E78-0460691682AC&deviationids%5B5%5D=34BF31C7-5019-A
4FB-43AC-EBBDE596B076&deviationids%5B6%5D=FF814C5A-9514-C713-EEC9-7D2C5FD53371&deviationids%5B7%5D=6DF734EB-763F-7F79-19C8-C26401373F7B&deviationids%5B8%5D=D7569B5C-4379-50EF-1AD3-4143FBD0B194&deviationids%5B9%5
D=74B3391C-CE0A-2997-CDF2-783E2E02B7B0&deviationids%5B10%5D=FF799C92-5D1F-21DC-C529-1D87046DBCDF&deviationids%5B11%5D=8201446E-C3A0-4544-863E-79B2DE8E7ED2&deviationids%5B12%5D=67DEF4C4-5713-A715-FB48-3413E3867EE
4&deviationids%5B13%5D=258EA935-1260-988C-6111-D53D4A200D83&deviationids%5B14%5D=C9B1895E-51E1-754D-BF7D-048A090D2046&deviationids%5B15%5D=0FE6EEB7-ED69-C1B6-3A24-47BEF2070B70&deviationids%5B16%5D=11A2FA65-5B22-
56CC-C955-86EFD6FB0D02&deviationids%5B17%5D=69CDC46C-B41B-CCB9-D494-8682B9CB230B&deviationids%5B18%5D=AED86DB2-4F87-98E1-76A9-D7C103EA5F05&deviationids%5B19%5D=5847262A-F4E5-5A78-ACC1-EBC3744011B1&deviationids%5
B20%5D=AE920495-917D-129E-B2DA-8650B865B09E&deviationids%5B21%5D=8922BB5E-3031-16ED-15F9-1E21FE015C19&deviationids%5B22%5D=F4D1FEC2-4D88-4096-B43A-ED3F782C0625&deviationids%5B23%5D=3963E09C-4865-AF0E-3328-B706F35A42E0&mature_content=true HTTP/1.1" 200 None
[deviantart][error] An unexpected error occurred: KeyError - 'category'. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues . 
[deviantart][debug]
Traceback (most recent call last):
  File "gallery_dl\job.pyc", line 96, in run
  File "gallery_dl\extractor\deviantart.pyc", line 112, in items
  File "gallery_dl\extractor\deviantart.pyc", line 751, in prepare
  File "gallery_dl\extractor\deviantart.pyc", line 197, in prepare
KeyError: 'category'

[mikf]

I hope I did nothing wrong.

You didn't, don't worry. If anything it was the DA devs who broke their API and caused the empty journal/status/literature issue to begin with, and me who implemented a bad and buggy workaround for it (#6196, #6207).

There is currently no fix. You could revert to v1.27.4 to get rid of the exception, but the downloaded literature items would all be empty.

@BlueSpiders99

[gallery-dl][debug] Version 1.25.8 - Executable

You are using an outdated version. This was fixed in v1.27.3 (269b2808)

[mikf]

Never mind, found the problem: https://github.com/mikf/gallery-dl/blob/f3f27496d6b084bb4defd94a2df34ec30fc9434c/gallery_dl/extractor/deviantart.py#L188 This needs to be "art" or DA complains:

{
  "error": "invalid_request",
  "errorDescription": "Unknown user/deviationid combination.",
  "errorCode": 404,
  "status": "error"
}

[sophi228]

Wow, congrats! I'm using the version directly from the GitHub using the Pip installation. Is there a way to update this myself or I will have to wait until the next update? Thanks in advance and I hope this isn't a dumb question

[sophi228]

Thanks for the fix! I tested with 2 different accounts and they both worked like a charm! <3

[mikf]

The solution from https://github.com/mikf/gallery-dl/commit/7dbd53e9b4ddb366d01b5405c117242292a6fb1f (get journal/literature HTML directly from webpage) should be better and more accurate than the previous __INITIAL_STATE__ markup.