Jules-WinnfieldX / CyberDropDownloader

Bulk Gallery Downloader for Cyberdrop.me and Other Sites
GNU General Public License v3.0
1.62k stars 182 forks source link

[BUG] `KeyError: 'f'` when scraping coomer #886

Closed baccccccc closed 5 months ago

baccccccc commented 5 months ago

note. I've only seen this for two URLs so far. The first URL looks in fact broken, visiting it with a browser gives you a 404. But the second URL looks good. At least, visiting it with a browser gives you an actual picture.

nevertheless, CDL gives you the same error for both URLs, and it's not even a 404. So, what I'm saying, it might be a symptom of a different issue.

https://img.coomer.party/thumbnail/data/15/b4/15b4091b846fe37cfb5610954521d02158d2b71ef0acc4077c441f45f280a674.jpg, See Log for Details
https://img.coomer.su/thumbnail/data/7d/ab/7dabf155b809c86240e265282c8771fff978afd708b0cda9b26e05844285984d.jpg, See Log for Details
INFO     : 2024-04-18 12:27:39,894 : utilities.py:103 : Scrape Starting: https://img.coomer.party/thumbnail/data/15/b4/15b4091b846fe37cfb5610954521d02158d2b71ef0acc4077c441f45f280a674.jpg
ERROR    : 2024-04-18 12:27:39,894 : utilities.py:103 : Scrape Failed: https://img.coomer.party/thumbnail/data/15/b4/15b4091b846fe37cfb5610954521d02158d2b71ef0acc4077c441f45f280a674.jpg ('f')
ERROR    : 2024-04-18 12:27:39,918 : utilities.py:103 : Traceback (most recent call last):
  File "C:\Users\<username>\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\cyberdrop_dl\utils\utilities.py", line 63, in wrapper
    return await func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\<username>\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\cyberdrop_dl\scraper\crawlers\coomer_crawler.py", line 99, in handle_direct_link
    filename, ext = await get_filename_and_ext(scrape_item.url.query["f"])
                                               ~~~~~~~~~~~~~~~~~~~~~^^^^^
KeyError: 'f'

INFO     : 2024-04-18 12:27:39,925 : utilities.py:103 : Scrape Finished: https://img.coomer.party/thumbnail/data/15/b4/15b4091b846fe37cfb5610954521d02158d2b71ef0acc4077c441f45f280a674.jpg

https://img.coomer.su/thumbnail/data/7d/ab/7dabf155b809c86240e265282c8771fff978afd708b0cda9b26e05844285984d.jpg
ERROR    : 2024-04-18 12:27:39,925 : utilities.py:103 : Scrape Failed: https://img.coomer.su/thumbnail/data/7d/ab/7dabf155b809c86240e265282c8771fff978afd708b0cda9b26e05844285984d.jpg ('f')
ERROR    : 2024-04-18 12:27:39,926 : utilities.py:103 : Traceback (most recent call last):
  File "C:\Users\<username>\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\cyberdrop_dl\utils\utilities.py", line 63, in wrapper
    return await func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\<username>\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\cyberdrop_dl\scraper\crawlers\coomer_crawler.py", line 99, in handle_direct_link
    filename, ext = await get_filename_and_ext(scrape_item.url.query["f"])
                                               ~~~~~~~~~~~~~~~~~~~~~^^^^^
KeyError: 'f'

INFO     : 2024-04-18 12:27:39,928 : utilities.py:103 : Scrape Finished: https://img.coomer.su/thumbnail/data/7d/ab/7dabf155b809c86240e265282c8771fff978afd708b0cda9b26e05844285984d.jpg
Jules-WinnfieldX commented 5 months ago

5.2.34 should fix