bitdruid / python-wayback-machine-downloader

Query and download archive.org as simple as possible.
MIT License
24 stars 1 forks source link

[BUG] #21

Open lostmagicblue opened 3 weeks ago

lostmagicblue commented 3 weeks ago

web: bbs.eastsea.com.cn question: waybackup -u http://bbs.eastsea.com.cn/ --workers 8 -c thank you !-- Exception: Worker: 4 - Exception !-- File: ../usr/lib64/python3.8/os.py !-- Function: makedirs !-- Line: 223 !-- Segment: mkdir(name, mode) !-- Description: [Errno 20] Not a directory: '/root/waybackup_snapshots/bbs.east sea.com.cn/nav/subjects/Y83Axianshua/2'

Exception log: /root/waybackup_snapshots/waybackup_error.log Exception in thread Thread-5: Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/pywaybackup/archive.py", line 294 , in download_loop download_status = download(output, snapshot, connection, statusmessage, no redirect) File "/usr/local/lib/python3.8/site-packages/pywaybackup/archive.py", line 364 , in download os.makedirs(output_path, exist_ok=True) File "/usr/lib64/python3.8/os.py", line 223, in makedirs mkdir(name, mode) NotADirectoryError: [Errno 20] Not a directory: '/root/waybackup_snapshots/bbs.e astsea.com.cn/nav/subjects/Y83Axianshua/2'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib64/python3.8/threading.py", line 932, in _bootstrap_inner self.run() File "/usr/lib64/python3.8/threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "/usr/local/lib/python3.8/site-packages/pywaybackup/archive.py", line 312 , in download_loop ex.exception(f"Worker: {worker} - Exception", e) File "/usr/local/lib/python3.8/site-packages/pywaybackup/Exception.py", line 7 6, in exception f.write(f" -- {var_name} = {value}\n") ValueError: I/O operation on closed file.

Tiptop4792 commented 2 weeks ago

I had this error too the other day. - It can occur when setting the -c option, since files are saved in the same directory instead of snapshot directories.

There seems to be a file called "2", and also maybe at a different time in history, there is a directory named 2.

--skip may solve this issue, but you'll loose one of the files.

bitdruid commented 2 weeks ago

web: bbs.eastsea.com.cn

question: waybackup -u http://bbs.eastsea.com.cn/ --workers 8 -c thank you !-- Exception: Worker: 4 - Exception !-- File: ../usr/lib64/python3.8/os.py !-- Function: makedirs !-- Line: 223 !-- Segment: mkdir(name, mode) !-- Description: [Errno 20] Not a directory: '/root/waybackup_snapshots/bbs.east sea.com.cn/nav/subjects/Y83Axianshua/2'

could you provide an error log for this? there should already be a functionality to handle paths and files with the same name colliding. or even better you could set the --log and then search for the collision manually so i can reproduce it :)