Jules-WinnfieldX / CyberDropDownloader

Bulk Gallery Downloader for Cyberdrop.me and Other Sites
GNU General Public License v3.0
1.61k stars 182 forks source link

[BUG] #763

Closed MachiavelliSeraphim closed 8 months ago

MachiavelliSeraphim commented 8 months ago

Describe the bug & To Help Reproduce Steps to help reproduce the behavior:

Having issues with Rule34.xxx Cannot find any links to download. Worked fine 3-4 weeks ago. Tried one link, didn't work, so I tried adding more thinking maybe something odd was going on. These 4 consistently will not download. Stopped trying new links afterwards.

https://rule34.xxx/index.php?page=post&s=list&tags=aerith_gainsborough https://rule34.xxx/index.php?page=post&s=list&tags=jessie_rasberry https://rule34.xxx/index.php?page=post&s=list&tags=tifa_lockhart https://rule34.xxx/index.php?page=post&s=list&tags=yuffie_kisaragi

Setup Information

Downloader.log file

INFO : 2023-12-31 11:30:52,758 : utilities.py:98 : Starting Async Processes... DEBUG : 2023-12-31 11:30:52,758 : utilities.py:98 : Starting Cyberdrop-DL Process for Default Config DEBUG : 2023-12-31 11:30:52,758 : utilities.py:98 : Running version 5.1.27 DEBUG : 2023-12-31 11:30:52,759 : utilities.py:98 : Using Config: Default DEBUG : 2023-12-31 11:30:52,759 : utilities.py:98 : Using Config File: AppData\Configs\Default\settings.yaml DEBUG : 2023-12-31 11:30:52,759 : utilities.py:98 : Using Input File: AppData\Configs\Default\URLs.txt DEBUG : 2023-12-31 11:30:52,759 : utilities.py:98 : Using Download Folder: Downloads\Cyberdrop-DL Downloads DEBUG : 2023-12-31 11:30:52,759 : utilities.py:98 : Using History File: AppData\Cache\cyberdrop.db DEBUG : 2023-12-31 11:30:52,759 : utilities.py:98 : Using Authentication: { "DDOS-Guard": { "Bunkrr Provided": false, "Coomer Provided": false, "Kemono Provided": false }, "Forums Credentials": { "celebforum Credentials Provided": false, "f95zone Credentials Provided": false, "leakedmodels Credentials Provided": false, "nudostar Credentials Provided": false, "simpcity Credentials Provided": false, "socialmediagirls Credentials Provided": false, "xbunker Credentials Provided": false }, "Forums XF Cookies": { "celebforum XF Cookie Provided": false, "f95zone XF Cookie Provided": false, "leakedmodels XF Cookie Provided": false, "nudostar XF Cookie Provided": false, "simpcity XF Cookie Provided": false, "socialmediagirls XF Cookie Provided": false, "xbunker XF Cookie Provided": false }, "GoFile Provided": false, "Imgur Provided": false, "JDownloader Provided": false, "PixelDrain Provided": false, "Reddit Provided": false } DEBUG : 2023-12-31 11:30:52,760 : utilities.py:98 : Using Settings: { "Download_Options": { "block_download_sub_folders": false, "disable_download_attempt_limit": false, "disable_file_timestamps": false, "include_album_id_in_folder_name": false, "include_thread_id_in_folder_name": false, "remove_domains_from_folder_names": false, "remove_generated_id_from_filenames": false, "scrape_single_forum_post": false, "separate_posts": false, "skip_download_mark_completed": false }, "File_Size_Limits": { "maximum_image_size": 0, "maximum_other_size": 0, "maximum_video_size": 0, "minimum_image_size": 0, "minimum_other_size": 0, "minimum_video_size": 0 }, "Files": { "download_folder": "Downloads\Cyberdrop-DL Downloads", "input_file": "AppData\Configs\Default\URLs.txt" }, "Ignore_Options": { "exclude_audio": false, "exclude_images": false, "exclude_other": false, "exclude_videos": false, "ignore_coomer_ads": false, "only_hosts": [], "skip_hosts": [] }, "Logs": { "download_error_urls_filename": "Download_Error_URLs.csv", "last_forum_post_filename": "Last_Scraped_Forum_Posts.txt", "log_folder": "AppData\Configs\Default\Logs", "main_log_filename": "downloader.log", "scrape_error_urls_filename": "Scrape_Error_URLs.csv", "unsupported_urls_filename": "Unsupported_URLs.txt" }, "Runtime_Options": { "delete_partial_files": false, "ignore_history": false, "log_level": 10, "send_unsupported_to_jdownloader": false, "skip_check_for_empty_folders": false, "skip_check_for_partial_files": false }, "Sorting": { "sort_downloads": false, "sort_folder": "Downloads\Cyberdrop-DL Sorted Downloads", "sort_incremementer_format": " ({i})", "sorted_audio": "{sort_dir}/{base_dir}/Audio/{filename}{ext}", "sorted_image": "{sort_dir}/{base_dir}/Images/{filename}{ext}", "sorted_other": "{sort_dir}/{base_dir}/Other/{filename}{ext}", "sorted_video": "{sort_dir}/{base_dir}/Videos/{filename}{ext}" } } DEBUG : 2023-12-31 11:30:52,760 : utilities.py:98 : Using Global Settings: { "General": { "allow_insecure_connections": true, "max_file_name_length": 95, "max_folder_name_length": 60, "proxy": "", "required_free_space": 5, "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/112.0" }, "Rate_Limiting_Options": { "connection_timeout": 15, "download_attempts": 10, "download_delay": 0.5, "max_simultaneous_downloads": 15, "max_simultaneous_downloads_per_domain": 4, "rate_limit": 50, "read_timeout": 300 } } INFO : 2023-12-31 11:30:53,268 : utilities.py:98 : Starting UI... WARNING : 2023-12-31 11:30:53,530 : utilities.py:98 : No valid links found. INFO : 2023-12-31 11:30:53,639 : utilities.py:98 : Running Post-Download Processes... INFO : 2023-12-31 11:30:53,639 : utilities.py:103 : Checking for partial downloads... INFO : 2023-12-31 11:30:53,641 : utilities.py:103 : Checking for empty folders... INFO : 2023-12-31 11:30:53,643 : utilities.py:98 : Printing Stats... INFO : 2023-12-31 11:30:53,643 : utilities.py:103 : Download Stats: INFO : 2023-12-31 11:30:53,644 : utilities.py:103 : Downloaded 0 files INFO : 2023-12-31 11:30:53,645 : utilities.py:103 : Previously Downloaded 0 files INFO : 2023-12-31 11:30:53,646 : utilities.py:103 : Skipped By Config 0 files INFO : 2023-12-31 11:30:53,647 : utilities.py:103 : Failed 0 files INFO : 2023-12-31 11:30:53,649 : utilities.py:103 : Scrape Failures: INFO : 2023-12-31 11:30:53,649 : utilities.py:103 : Download Failures: INFO : 2023-12-31 11:30:53,650 : utilities.py:98 : Checking for Program End... INFO : 2023-12-31 11:30:53,650 : utilities.py:98 : Checking for Updates... INFO : 2023-12-31 11:30:53,886 : utilities.py:98 : Closing Program... INFO : 2023-12-31 11:30:53,888 : utilities.py:103 : Finished downloading. Enjoy :)

Jules-WinnfieldX commented 8 months ago

You didn't put the links in the urls file which is here: .\AppData\Configs\Default\URLs.txt

MachiavelliSeraphim commented 8 months ago

My bad.

I upgraded from an old version and didn't realize the URLs.txt moved. The old location works fine on other sites which is wierd....

Thanks. Sorry for the trouble.

MachiavelliSeraphim commented 8 months ago

Still doesnt download anything :(

INFO : 2023-12-31 11:39:27,263 : utilities.py:98 : Starting Async Processes... DEBUG : 2023-12-31 11:39:27,264 : utilities.py:98 : Starting Cyberdrop-DL Process for Default Config DEBUG : 2023-12-31 11:39:27,264 : utilities.py:98 : Running version 5.1.27 DEBUG : 2023-12-31 11:39:27,264 : utilities.py:98 : Using Config: Default DEBUG : 2023-12-31 11:39:27,264 : utilities.py:98 : Using Config File: AppData\Configs\Default\settings.yaml DEBUG : 2023-12-31 11:39:27,264 : utilities.py:98 : Using Input File: AppData\Configs\Default\URLs.txt DEBUG : 2023-12-31 11:39:27,264 : utilities.py:98 : Using Download Folder: Downloads\Cyberdrop-DL Downloads DEBUG : 2023-12-31 11:39:27,264 : utilities.py:98 : Using History File: AppData\Cache\cyberdrop.db DEBUG : 2023-12-31 11:39:27,265 : utilities.py:98 : Using Authentication: { "DDOS-Guard": { "Bunkrr Provided": false, "Coomer Provided": false, "Kemono Provided": false }, "Forums Credentials": { "celebforum Credentials Provided": false, "f95zone Credentials Provided": false, "leakedmodels Credentials Provided": false, "nudostar Credentials Provided": false, "simpcity Credentials Provided": false, "socialmediagirls Credentials Provided": false, "xbunker Credentials Provided": false }, "Forums XF Cookies": { "celebforum XF Cookie Provided": false, "f95zone XF Cookie Provided": false, "leakedmodels XF Cookie Provided": false, "nudostar XF Cookie Provided": false, "simpcity XF Cookie Provided": false, "socialmediagirls XF Cookie Provided": false, "xbunker XF Cookie Provided": false }, "GoFile Provided": false, "Imgur Provided": false, "JDownloader Provided": false, "PixelDrain Provided": false, "Reddit Provided": false } DEBUG : 2023-12-31 11:39:27,265 : utilities.py:98 : Using Settings: { "Download_Options": { "block_download_sub_folders": false, "disable_download_attempt_limit": false, "disable_file_timestamps": false, "include_album_id_in_folder_name": false, "include_thread_id_in_folder_name": false, "remove_domains_from_folder_names": false, "remove_generated_id_from_filenames": false, "scrape_single_forum_post": false, "separate_posts": false, "skip_download_mark_completed": false }, "File_Size_Limits": { "maximum_image_size": 0, "maximum_other_size": 0, "maximum_video_size": 0, "minimum_image_size": 0, "minimum_other_size": 0, "minimum_video_size": 0 }, "Files": { "download_folder": "Downloads\Cyberdrop-DL Downloads", "input_file": "AppData\Configs\Default\URLs.txt" }, "Ignore_Options": { "exclude_audio": false, "exclude_images": false, "exclude_other": false, "exclude_videos": false, "ignore_coomer_ads": false, "only_hosts": [], "skip_hosts": [] }, "Logs": { "download_error_urls_filename": "Download_Error_URLs.csv", "last_forum_post_filename": "Last_Scraped_Forum_Posts.txt", "log_folder": "AppData\Configs\Default\Logs", "main_log_filename": "downloader.log", "scrape_error_urls_filename": "Scrape_Error_URLs.csv", "unsupported_urls_filename": "Unsupported_URLs.txt" }, "Runtime_Options": { "delete_partial_files": false, "ignore_history": false, "log_level": 10, "send_unsupported_to_jdownloader": false, "skip_check_for_empty_folders": false, "skip_check_for_partial_files": false }, "Sorting": { "sort_downloads": false, "sort_folder": "Downloads\Cyberdrop-DL Sorted Downloads", "sort_incremementer_format": " ({i})", "sorted_audio": "{sort_dir}/{base_dir}/Audio/{filename}{ext}", "sorted_image": "{sort_dir}/{base_dir}/Images/{filename}{ext}", "sorted_other": "{sort_dir}/{base_dir}/Other/{filename}{ext}", "sorted_video": "{sort_dir}/{base_dir}/Videos/{filename}{ext}" } } DEBUG : 2023-12-31 11:39:27,265 : utilities.py:98 : Using Global Settings: { "General": { "allow_insecure_connections": true, "max_file_name_length": 95, "max_folder_name_length": 60, "proxy": "", "required_free_space": 5, "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/112.0" }, "Rate_Limiting_Options": { "connection_timeout": 15, "download_attempts": 10, "download_delay": 0, "max_simultaneous_downloads": 15, "max_simultaneous_downloads_per_domain": 4, "rate_limit": 50, "read_timeout": 300 } } INFO : 2023-12-31 11:39:27,271 : utilities.py:98 : Starting UI... INFO : 2023-12-31 11:39:27,533 : utilities.py:98 : Scrape Starting: https://rule34.xxx/index.php?page=post&s=list&tags=aerith_gainsborough INFO : 2023-12-31 11:39:28,044 : utilities.py:98 : Scrape Finished: https://rule34.xxx/index.php?page=post&s=list&tags=aerith_gainsborough DEBUG : 2023-12-31 11:39:28,045 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=list&tags=jessie_rasberry as it has already been scraped DEBUG : 2023-12-31 11:39:28,045 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=list&tags=tifa_lockhart as it has already been scraped DEBUG : 2023-12-31 11:39:28,045 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=list&tags=yuffie_kisaragi as it has already been scraped DEBUG : 2023-12-31 11:39:28,045 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9264464 as it has already been scraped DEBUG : 2023-12-31 11:39:28,046 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9263817 as it has already been scraped DEBUG : 2023-12-31 11:39:28,046 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9260300 as it has already been scraped DEBUG : 2023-12-31 11:39:28,046 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9254121 as it has already been scraped DEBUG : 2023-12-31 11:39:28,046 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9253335 as it has already been scraped DEBUG : 2023-12-31 11:39:28,046 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9252704 as it has already been scraped DEBUG : 2023-12-31 11:39:28,046 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9252703 as it has already been scraped DEBUG : 2023-12-31 11:39:28,047 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9246623 as it has already been scraped DEBUG : 2023-12-31 11:39:28,047 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9245159 as it has already been scraped DEBUG : 2023-12-31 11:39:28,047 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9245154 as it has already been scraped DEBUG : 2023-12-31 11:39:28,047 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9240275 as it has already been scraped DEBUG : 2023-12-31 11:39:28,047 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9239756 as it has already been scraped DEBUG : 2023-12-31 11:39:28,047 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9236878 as it has already been scraped DEBUG : 2023-12-31 11:39:28,047 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9235782 as it has already been scraped DEBUG : 2023-12-31 11:39:28,048 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9233662 as it has already been scraped DEBUG : 2023-12-31 11:39:28,048 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9230134 as it has already been scraped DEBUG : 2023-12-31 11:39:28,048 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9228539 as it has already been scraped DEBUG : 2023-12-31 11:39:28,048 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9227133 as it has already been scraped DEBUG : 2023-12-31 11:39:28,048 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9224628 as it has already been scraped DEBUG : 2023-12-31 11:39:28,048 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9224518 as it has already been scraped DEBUG : 2023-12-31 11:39:28,048 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9218937 as it has already been scraped DEBUG : 2023-12-31 11:39:28,049 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9218932 as it has already been scraped DEBUG : 2023-12-31 11:39:28,049 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9216122 as it has already been scraped DEBUG : 2023-12-31 11:39:28,049 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9211691 as it has already been scraped DEBUG : 2023-12-31 11:39:28,049 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9211515 as it has already been scraped DEBUG : 2023-12-31 11:39:28,049 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9211513 as it has already been scraped DEBUG : 2023-12-31 11:39:28,049 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9211499 as it has already been scraped DEBUG : 2023-12-31 11:39:28,049 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9211488 as it has already been scraped DEBUG : 2023-12-31 11:39:28,050 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9210728 as it has already been scraped DEBUG : 2023-12-31 11:39:28,050 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9209851 as it has already been scraped DEBUG : 2023-12-31 11:39:28,050 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9207504 as it has already been scraped DEBUG : 2023-12-31 11:39:28,050 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9207327 as it has already been scraped DEBUG : 2023-12-31 11:39:28,050 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9205905 as it has already been scraped DEBUG : 2023-12-31 11:39:28,050 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9205387 as it has already been scraped DEBUG : 2023-12-31 11:39:28,050 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9205371 as it has already been scraped DEBUG : 2023-12-31 11:39:28,051 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9205363 as it has already been scraped DEBUG : 2023-12-31 11:39:28,051 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9205360 as it has already been scraped DEBUG : 2023-12-31 11:39:28,051 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9205069 as it has already been scraped DEBUG : 2023-12-31 11:39:28,051 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9203973 as it has already been scraped DEBUG : 2023-12-31 11:39:28,051 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9203968 as it has already been scraped DEBUG : 2023-12-31 11:39:28,051 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9203958 as it has already been scraped DEBUG : 2023-12-31 11:39:28,051 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=view&id=9201953 as it has already been scraped DEBUG : 2023-12-31 11:39:28,052 : utilities.py:98 : Skipping https://rule34.xxx/index.php?page=post&s=list&tags=aerith_gainsborough&pid=42 as it has already been scraped INFO : 2023-12-31 11:39:28,196 : utilities.py:98 : Running Post-Download Processes... INFO : 2023-12-31 11:39:28,196 : utilities.py:103 : Checking for partial downloads... INFO : 2023-12-31 11:39:28,200 : utilities.py:103 : Checking for empty folders... INFO : 2023-12-31 11:39:28,206 : utilities.py:98 : Printing Stats... INFO : 2023-12-31 11:39:28,207 : utilities.py:103 : Download Stats: INFO : 2023-12-31 11:39:28,209 : utilities.py:103 : Downloaded 0 files INFO : 2023-12-31 11:39:28,211 : utilities.py:103 : Previously Downloaded 0 files INFO : 2023-12-31 11:39:28,214 : utilities.py:103 : Skipped By Config 0 files INFO : 2023-12-31 11:39:28,217 : utilities.py:103 : Failed 0 files INFO : 2023-12-31 11:39:28,218 : utilities.py:103 : Scrape Failures: INFO : 2023-12-31 11:39:28,219 : utilities.py:103 : Download Failures: INFO : 2023-12-31 11:39:28,220 : utilities.py:98 : Checking for Program End... INFO : 2023-12-31 11:39:28,220 : utilities.py:98 : Checking for Updates... INFO : 2023-12-31 11:39:28,438 : utilities.py:98 : Closing Program... INFO : 2023-12-31 11:39:28,438 : utilities.py:103 : Finished downloading. Enjoy :)

Jules-WinnfieldX commented 8 months ago

hm. I'll look into this later.

MachiavelliSeraphim commented 8 months ago

I just reinstalled cyberdrop from scratch.
Ill give it one more go and let you know in a few minutes. Maybe I had a corrupted install.

MachiavelliSeraphim commented 8 months ago

Same error log after reinstall.

Thanks.

Jules-WinnfieldX commented 8 months ago

5.1.31 should fix this.