Jules-WinnfieldX / CyberDropDownloader

Bulk Gallery Downloader for Cyberdrop.me and Other Sites
GNU General Public License v3.0
1.64k stars 189 forks source link

Seems to finish without downloading any files #516

Closed Russellva closed 1 year ago

Russellva commented 1 year ago

Hi,

I'm just looking for feedback here. I'm a beginner in the world of coding for the most part.

I've been using the cyberdrop-dl for a number of months now with no problems what so ever. However, recently, I can't seem to get it to work how it was prior. I had been primarily using some other code for a period of time, and coming back to use cyberdrop-dl after a couple weeks, I can't get it to download anymore.

I have tried numerous different links over a period of time and always get the same result. It scans the link, starts them, and then returns completed with zero files shortly after.

I've changed the directory to be in the correct location, but it does not seem to be downloading anything.

Any feedback would be greatly appreciated!

Jules-WinnfieldX commented 1 year ago

I'd need to see the downloader.log file after what you described happened, can't tell you anything without it

Russellva commented 1 year ago

These are the last few lines. The previous lines show the individual links which are trying to be downloaded. It completes, but there's nothing in the output folder. ... 2023-07-13 10:32:38,504:DEBUG:base_functions:base_functions.py:56:| [green]Files Complete: 0[/green] - [yellow]Files Skipped: 59[/yellow] - [red]Files Failed: 0[/red] | 2023-07-13 10:32:38,506:DEBUG:base_functions:base_functions.py:56: 2023-07-13 10:32:38,506:DEBUG:base_functions:base_functions.py:56:Checking for incomplete downloads 2023-07-13 10:32:38,507:DEBUG:base_functions:base_functions.py:56:Purging empty directories 2023-07-13 10:32:38,508:DEBUG:base_functions:base_functions.py:56:Finished downloading. Enjoy :) 2023-07-13 10:32:38,753:ERROR:base_events:base_events.py:1752:Unclosed client session client_session: <aiohttp.client.ClientSession object at 0x7fb4c8b95210> 2023-07-13 10:32:38,754:ERROR:base_events:base_events.py:1752:Unclosed connector connections: ['[(<aiohttp.client_proto.ResponseHandler object at 0x7fb4c8c71780>, 220748.910854042)]'] connector: <aiohttp.connector.TCPConnector object at 0x7fb4c8b95390> 2023-07-13 10:32:38,755:DEBUG:base_functions:base_functions.py:56: If you enjoy using this program, please consider buying the developer a coffee :) https://www.buymeacoffee.com/juleswinnft 2023-07-13 10:32:38,758:INFO:aiorun:aiorun.py:263:Entering shutdown phase. 2023-07-13 10:32:38,758:INFO:aiorun:aiorun.py:359:Waiting for executor shutdown. 2023-07-13 10:32:38,758:INFO:aiorun:aiorun.py:363:Shutting down async generators 2023-07-13 10:32:38,758:INFO:aiorun:aiorun.py:365:Closing the loop. 2023-07-13 10:32:38,759:INFO:aiorun:aiorun.py:367:Leaving. Bye! 2023-07-13 10:32:38,760:DEBUG:sql_helper:sql_helper.py:281:Successfully closed sqlite database connection

Jules-WinnfieldX commented 1 year ago

So I'd need to see the entire thing.

But it skipped the links you gave it, most likely because you had already downloaded them in the past. You can disable download history in the config.

reynarone commented 1 year ago

Sorry to hijack thread, i also face this same issues.

I also updated to latest version and ran with --ignore-history. This only happens for jpg.fish sites, bunkrr is fine for me

2023-08-25 19:06:35,464:DEBUG:proactor_events:proactor_events.py:633:Using proactor: IocpProactor
2023-08-25 19:06:35,495:DEBUG:base_functions:base_functions.py:51:Starting Cyberdrop-DL
2023-08-25 19:06:35,519:DEBUG:base_functions:base_functions.py:51:Using authentication arguments: {'gofile_api_key': '', 'pixeldrain_api_key': '!REDACTED!', 'nudostar_username': '', 'nudostar_password': '!REDACTED!', 'simpcity_username': '', 'simpcity_password': '!REDACTED!', 'socialmediagirls_username': '', 'socialmediagirls_password': '!REDACTED!', 'xbunker_username': '', 'xbunker_password': '!REDACTED!'}
2023-08-25 19:06:35,519:DEBUG:base_functions:base_functions.py:51:Using file arguments: {'db_file': WindowsPath('download_history.sqlite'), 'errored_urls_file': WindowsPath('Errored_URLs.csv'), 'input_file': WindowsPath('URLs.txt'), 'log_file': WindowsPath('downloader.log'), 'output_folder': WindowsPath('Downloads'), 'output_last_forum_post_file': WindowsPath('URLs_last_post.txt'), 'unsupported_urls_file': WindowsPath('Unsupported_URLs.csv')}
2023-08-25 19:06:35,520:DEBUG:base_functions:base_functions.py:51:Using forum option arguments: {'output_last_forum_post': False, 'scrape_single_post': False, 'separate_posts': False}
2023-08-25 19:06:35,520:DEBUG:base_functions:base_functions.py:51:Using ignore arguments: {'exclude_videos': False, 'exclude_images': False, 'exclude_audio': False, 'exclude_other': False, 'ignore_cache': False, 'ignore_history': True, 'skip_hosts': [], 'only_hosts': []}
2023-08-25 19:06:35,520:DEBUG:base_functions:base_functions.py:51:Using jdownloader arguments: {'apply_jdownloader': False, 'jdownloader_username': '', 'jdownloader_password': '!REDACTED!', 'jdownloader_device': ''}
2023-08-25 19:06:35,520:DEBUG:base_functions:base_functions.py:51:Using progress option arguments: {'hide_new_progress': False, 'hide_overall_progress': False, 'hide_forum_progress': False, 'hide_thread_progress': False, 'hide_domain_progress': False, 'hide_album_progress': False, 'hide_file_progress': False, 'refresh_rate': 10, 'visible_rows_threads': 2, 'visible_rows_domains': 2, 'visible_rows_albums': 2, 'visible_rows_files': 10}
2023-08-25 19:06:35,520:DEBUG:base_functions:base_functions.py:51:Using ratelimiting arguments: {'connection_timeout': 15, 'ratelimit': 50, 'throttle': 0.5}
2023-08-25 19:06:35,520:DEBUG:base_functions:base_functions.py:51:Using runtime arguments: {'allow_insecure_connections': False, 'attempts': 10, 'block_sub_folders': False, 'disable_attempt_limit': False, 'include_id': False, 'output_errored_urls': False, 'output_unsupported_urls': False, 'proxy': '', 'remove_bunkr_identifier': False, 'required_free_space': 5, 'simultaneous_downloads_per_domain': 4, 'skip_download_mark_completed': False, 'user_agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/112.0'}
2023-08-25 19:06:35,520:DEBUG:base_functions:base_functions.py:51:Using sorting arguments: {'sort_downloads': False, 'sort_directory': WindowsPath('Sorted Downloads'), 'sorted_audio': '{sort_dir}/{base_dir}/Audio', 'sorted_images': '{sort_dir}/{base_dir}/Images', 'sorted_others': '{sort_dir}/{base_dir}/Other', 'sorted_videos': '{sort_dir}/{base_dir}/Videos'}
2023-08-25 19:06:35,521:DEBUG:base_functions:base_functions.py:51:We are running version 4.2.21 of Cyberdrop Downloader
2023-08-25 19:06:35,620:DEBUG:base_functions:base_functions.py:51:Starting Scrape
2023-08-25 19:06:35,621:DEBUG:base_functions:base_functions.py:51:Starting: https://jpg.fish/a/aqua-no158.cVRdo/?sort=date_desc&page=1
2023-08-25 19:06:35,626:DEBUG:base_functions:base_functions.py:51:Starting: https://jpg.fish/a/sweet-dream.kU0jp
2023-08-25 19:06:35,628:DEBUG:base_functions:base_functions.py:51:Starting: https://jpg.fish/a/yuuhui-no001.Plra9
2023-08-25 19:06:35,630:DEBUG:base_functions:base_functions.py:51:Starting: https://jpg.fish/a/yuuhui-yuhui-badnjak-odbjegli-jelen110p-1v-12g-telegraph.PlxjI
2023-08-25 19:06:35,631:DEBUG:base_functions:base_functions.py:51:Starting: https://jpg.fish/a/34p344mb.kUnIK
2023-08-25 19:06:35,633:DEBUG:base_functions:base_functions.py:51:Starting: https://jpg.fish/a/96p1v117mb.PoKQe
2023-08-25 19:06:35,634:DEBUG:base_functions:base_functions.py:51:Starting: https://jpg.fish/a/aqua-nilu-genshin-impact.cuWiw
2023-08-25 19:06:35,637:DEBUG:base_functions:base_functions.py:51:Starting: https://jpg.fish/a/byoru-ada-wong.yQA6u
2023-08-25 19:06:35,640:DEBUG:base_functions:base_functions.py:51:Starting: https://jpg.fish/a/byoru-lucy.BXhPK
2023-08-25 19:06:38,292:DEBUG:base_functions:base_functions.py:51:Finished: https://jpg.fish/a/34p344mb.kUnIK
2023-08-25 19:06:38,361:DEBUG:base_functions:base_functions.py:51:Finished: https://jpg.fish/a/aqua-nilu-genshin-impact.cuWiw
2023-08-25 19:06:38,499:DEBUG:base_functions:base_functions.py:51:Finished: https://jpg.fish/a/yuuhui-no001.Plra9
2023-08-25 19:06:38,578:DEBUG:base_functions:base_functions.py:51:Finished: https://jpg.fish/a/byoru-lucy.BXhPK
2023-08-25 19:06:38,656:DEBUG:base_functions:base_functions.py:51:Finished: https://jpg.fish/a/sweet-dream.kU0jp
2023-08-25 19:06:39,090:DEBUG:base_functions:base_functions.py:51:Finished: https://jpg.fish/a/byoru-ada-wong.yQA6u
2023-08-25 19:06:39,131:DEBUG:base_functions:base_functions.py:51:Finished: https://jpg.fish/a/aqua-no158.cVRdo/?sort=date_desc&page=1
2023-08-25 19:06:39,777:DEBUG:base_functions:base_functions.py:51:Finished: https://jpg.fish/a/yuuhui-yuhui-badnjak-odbjegli-jelen110p-1v-12g-telegraph.PlxjI
2023-08-25 19:06:40,018:DEBUG:base_functions:base_functions.py:51:Finished: https://jpg.fish/a/96p1v117mb.PoKQe
2023-08-25 19:06:40,020:DEBUG:base_functions:base_functions.py:51:
2023-08-25 19:06:40,020:DEBUG:base_functions:base_functions.py:51:Finished Scrape
2023-08-25 19:06:45,070:DEBUG:base_functions:base_functions.py:51:
2023-08-25 19:06:45,071:DEBUG:base_functions:base_functions.py:51:Checking for incomplete downloads
2023-08-25 19:06:45,077:DEBUG:base_functions:base_functions.py:51:Purging empty directories
2023-08-25 19:06:45,090:DEBUG:base_functions:base_functions.py:51:Finished downloading. Enjoy :)
2023-08-25 19:06:45,091:DEBUG:base_functions:base_functions.py:51:There are partial downloads in the downloads folder.
2023-08-25 19:06:45,092:DEBUG:base_functions:base_functions.py:51:
2023-08-25 19:06:45,093:DEBUG:base_functions:base_functions.py:51:If you enjoy using this program, please consider buying the developer a coffee :)
https://www.buymeacoffee.com/juleswinnft
2023-08-25 19:06:45,095:DEBUG:sql_helper:sql_helper.py:281:Successfully closed sqlite database connection
Jules-WinnfieldX commented 1 year ago

Sorry to hijack thread, i also face this same issues.

I also updated to latest version and ran with --ignore-history. This only happens for jpg.fish sites, bunkrr is fine for me

2023-08-25 19:06:35,521:DEBUG:base_functions:base_functions.py:51:We are running version 4.2.21 of Cyberdrop Downloader

you need to update

reynarone commented 1 year ago

Ok, it works now

PS E:\Tools\Cyberdrop_DL.V3> pip show cyberdrop-dl
Name: cyberdrop-dl
Version: 4.2.21
Summary: Bulk downloader for multiple file hosts and forum sites
Home-page: https://github.com/Jules-WinnfieldX/CyberDropDownloader
Author: Jules-WinnfieldX
Author-email: JulesWinnfieldII@protonmail.com
License:
Location: C:\Python311\Lib\site-packages
Requires: aiofiles, aiohttp, aiolimiter, beautifulsoup4, certifi, myjdapi, PyYAML, rich, setuptools, tqdm, yarl
Required-by:
PS E:\Tools\Cyberdrop_DL.V3> cyberdrop-dl
cyberdrop-dl : The term 'cyberdrop-dl' is not recognized as the name of a cmdlet, function, script file, or operable
program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ cyberdrop-dl
+ ~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (cyberdrop-dl:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

Suggestion [3,General]: The command cyberdrop-dl was not found, but does exist in the current location. Windows PowerShell does not load commands from the current location by default. If you trust this command, instead type: ".\cyberdrop-dl". See "get-help about_Command_Precedence" for more details.
PS E:\Tools\Cyberdrop_DL.V3> .\cyberdrop-dl.exe
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "E:\Tools\Cyberdrop_DL.V3\cyberdrop-dl.exe\__main__.py", line 4, in <module>
ModuleNotFoundError: No module named 'cyberdrop_dl'
PS E:\Tools\Cyberdrop_DL.V3> cyberdrop-dl -V
cyberdrop-dl 4.2.189
PS E:\Tools\Cyberdrop_DL.V3> pip show cyberdrop-dl
Name: cyberdrop-dl
Version: 4.2.189
Summary: Bulk downloader for multiple file hosts and forum sites
Home-page: https://github.com/Jules-WinnfieldX/CyberDropDownloader
Author: Jules-WinnfieldX
Author-email: JulesWinnfieldII@protonmail.com
License:
Location: C:\Python311\Lib\site-packages
Requires: aiofiles, aiohttp, aiolimiter, aiorun, asyncpraw, beautifulsoup4, certifi, myjdapi, platformdirs, PyYAML, rich, setuptools, tqdm, yarl
Required-by:
PS E:\Tools\Cyberdrop_DL.V3> cyberdrop-dl -V
cyberdrop-dl 4.2.189
PS E:\Tools\Cyberdrop_DL.V3>

Somehow fixed by going thru the .bat file instead of using powershell

Jules-WinnfieldX commented 1 year ago

For those that may eventually look at this, it's very likely the above person has multiple versions of python installed, and that's causing issues.