search

alexgand / springer_free_books

Python script to download all Springer books released for free during the 2020 COVID-19 quarantine
GNU General Public License v3.0
1.65k stars 367 forks source link

Download fails around 4%, "File name too long" & NameError: name 'time' is not defined #72

Closed jesusgt closed 4 years ago

jesusgt commented 4 years ago

Python version: 3.7.2 OS: Ubuntu 16.04.5 LTS

I finally managed to get the script going, however after some downloads (aroud 4%) the script throws up a bunch of errors and exits, leaving the download half-way. Re-running the script results in the same issue repeatedly.

Any help in resolving this issue will be much appreciated. Thanks in advance.

`[Errno 36] File name too long: './downloads/Medicine/Primer on the Rheumatic Diseases - John H. Klippel, John H. Stone, L eslie J. Crofford, Patience H. Whi>

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/xuxo/dl/videos/_descargados_a_ordenar/Springer_Books/springer_free_books/helper.py", line 113, in download_selected_books download_book(request, output_file, patch) File "/home/xuxo/dl/videos/_descargados_a_ordenar/Springer_Books/springer_free_books/helper.py", line 90, in download_book _download_book(new_url, output_file) File "/home/xuxo/dl/videos/_descargados_a_ordenar/Springer_Books/springer_free_books/helper.py", line 83, in _download_book shutil.move(tmp_file, book_path) File "/usr/lib/python3.7/shutil.py", line 580, in move copy_function(src, real_dst) File "/usr/lib/python3.7/shutil.py", line 266, in copy2 copyfile(src, dst, follow_symlinks=follow_symlinks) File "/usr/lib/python3.7/shutil.py", line 121, in copyfile with open(dst, 'wb') as fdst: OSError: [Errno 36] File name too long: './downloads/Medicine/Primer on the Rheumatic Diseases - John H. Klippel, John H. Stone, L eslie J. Crofford, Patien>

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "main.py", line 73, in download_selected_books(books, folder, patches) File "/home/xuxo/dl/videos/_descargados_a_ordenar/Springer_Books/springer_free_books/helper.py", line 119, in download_selected_books time.sleep(30) NameError: name 'time' is not defined

`

chaosAD commented 4 years ago

Try this temporary project here.

I have added a new option where you can specify the maximum filename length using -m option. The command below sets it to a maximum of 100 and downloads only the Primer on the Rheumatic Diseases book.

python main.py -m 100 -i 17

If you omit the option, it defaults to 145. Please try to find the best length that works. Let me know.

jesusgt commented 4 years ago

Sorry ChaosAD. I think I spoke too early. After cloning from the temporary project (https://github.com/chaosAD/springer_free_books.git), and trying the -m option, the script throws out an error about not recognising the "-m" argument. There's a different behaviour from the new script though, it continues downloading onto the next book when it encounters one with a name too long (the old one did not).

`(.venv)[BaCKuPS][~/springer_free_books]$ python main.py -m 120 -i 182 184 261 310 311 17

usage: main.py [-h] [-f FOLDER] [--pdf] [--epub] [-c CATEGORY [CATEGORY ...]] [-i BOOK_INDEX [BOOK_INDEX ...]] [-v]

main.py: error: unrecognized arguments: -m 120 `

chaosAD commented 4 years ago

You cloned from the wrong branch. You should have used the link I gave you. But anyway, I have fixed this issue in PR #79. Since it needs time for Alex to accept my PR, I suggest you clone it from my repository below: https://github.com/chaosAD/springer_free_books/tree/95380990cc5c0db9b05308f9f930ea433e10884b

The new code determine the maximum filename length automatically, so you don't issue the -m option anymore. Let me know if you encounter anymore issue.

Note: If you are downloading books by indices (-i option), you need to take note that the index numbering now is different than the older Python script. Check this PR #74 to learn more.

agkini commented 4 years ago

Thanks to whoever fixed this, unfortunately looks like this code is reverted because it wasn't working in Python3?

I got: TypeError: Unicode-objects must be encoded before hashing

Someone reverted it a few minutes ago.

chaosAD commented 4 years ago

Yes, Alex reverted it because it wasn't working on Python3. He showed me the same error as you did. Could you try this commit? Let me know if it works.

jesusgt commented 4 years ago

Still no luck here with the last commit suggested:

`(.venv) [BaCKuPS][~/springer_free_books]$ python main.py

407 titles ready to be downloaded... Traceback (most recent call last): File "main.py", line 72, in download_books(books, folder, patches) File "/home/xuxo/springer_free_books/helper.py", line 98, in download_books max_length = get_max_filename_length(folder) File "/home/xuxo/springer_free_books/helper.py", line 163, in get_max_filename_length name = get_random_string(mid) File "/home/xuxo/springer_free_books/helper.py", line 153, in get_random_string for i in range(0, (length / 128 + 1)): TypeError: 'float' object cannot be interpreted as an integer `

chaosAD commented 4 years ago

I have just fixed that issue. Try this latest commit.

jesusgt commented 4 years ago

Hi there. Many thanks again chaosAD. All working here after the latest commit.