jbsparrow / CyberDropDownloader

Bulk Gallery Downloader for Cyberdrop.me and Other Sites
GNU General Public License v3.0
156 stars 13 forks source link

[BUG] `index out of range` on cyberfile.su in 5.7.0rc2 #199

Closed baccccccc closed 8 hours ago

baccccccc commented 9 hours ago
[10/23/24 12:12:03] INFO     Scrape Starting:                                                                                                   utilities.py:112
                             https://cyberfile.su/folder/0d820cc5fadc53ba3f1cf98ca3fd0a62/Alexa_Pearl_(Updated_May_22,_2022)_by_Chamaleon                       

                    ERROR    Scrape Failed:                                                                                                     utilities.py:112
                             https://cyberfile.su/folder/0d820cc5fadc53ba3f1cf98ca3fd0a62/Alexa_Pearl_(Updated_May_22,_2022)_by_Chamaleon (list                 
                             index out of range)                                                                                                                
                             ╭────────────────────────────────────── Traceback (most recent call last) ───────────────────────────────────────╮                 
                             │ C:\Users\<username>\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-pack │                 
                             │ ages\Python312\site-packages\cyberdrop_dl\utils\utilities.py:71 in wrapper                                     │                 
                             │                                                                                                                │                 
                             │    68 │   │   link = args[0] if isinstance(args[0], URL) else args[0].url                                      │                 
                             │    69 │   │                                                                                                    │                 
                             │    70 │   │   try:                                                                                             │                 
                             │ ❱  71 │   │   │   return await func(self, *args, **kwargs)                                                     │                 
                             │    72 │   │   except NoExtensionFailure:                                                                       │                 
                             │    73 │   │   │   await log(f"Scrape Failed: {link} (No File Extension)", 40)                                  │                 
                             │    74 │   │   │   await self.manager.log_manager.write_scrape_error_log(link, " No File                        │                 
                             │       Extension")                                                                                              │                 
                             │                                                                                                                │                 
                             │ C:\Users\<username>\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-pack │                 
                             │ ages\Python312\site-packages\cyberdrop_dl\scraper\crawlers\cyberfile_crawler.py:49 in folder                   │                 
                             │                                                                                                                │                 
                             │    46 │   │   async with self.request_limiter:                                                                 │                 
                             │    47 │   │   │   soup: BeautifulSoup = await self.client.get_BS4(self.domain,                                 │                 
                             │       scrape_item.url)                                                                                         │                 
                             │    48 │   │                                                                                                    │                 
                             │ ❱  49 │   │   script_func = soup.select('div[class*="page-container"] script')[-1].text                        │                 
                             │    50 │   │   script_func = script_func.split('loadImages(')[-1]                                               │                 
                             │    51 │   │   script_func = script_func.split(';')[0]                                                          │                 
                             │    52 │   │   nodeId = int(script_func.split(',')[1].replace("'", ""))                                         │                 
                             ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯                 
                             IndexError: list index out of range                                                                                                

                    INFO     Scrape Finished:                                                                                                   utilities.py:112
                             https://cyberfile.su/folder/0d820cc5fadc53ba3f1cf98ca3fd0a62/Alexa_Pearl_(Updated_May_22,_2022)_by_Chamaleon                       
jbsparrow commented 9 hours ago

Looks like it's an issue with handling deleted items, will take a look at fixing it in a little bit.

jbsparrow commented 8 hours ago

Fixed in 5.7.0rc3.