ferru97 / PyPaperBot

PyPaperBot is a Python tool for downloading scientific papers using Google Scholar, Crossref, SciHub, and SciDB.
MIT License
393 stars 74 forks source link

Skip the DOI #27

Open mbsouksu opened 3 years ago

mbsouksu commented 3 years ago

Hello,

Thank you for your tool. It is magnificent and very useful. I just want to highlight a minor thing. When it is searching for the list of DOIs, if it can't find it, this causes an error when it comes to download it and stop the program.

ferru97 commented 3 years ago

Hi!

Thank you for your interest! Can you post a log of the error?

kir-malishev commented 3 years ago

I think this is due to the fact that when the article is not found, the return value of the function getPapersInfoFromDOIs is not processed in any special way. It might be worth returning None from getPapersInfoFromDOIs and checking it at https://github.com/ferru97/PyPaperBot/blob/ee5b5020cb39396835ae7805483c3516169f3268/PyPaperBot/__main__.py#L25

mbsouksu commented 3 years ago

I guess @kir-malishev is right about the error and here the error I got.

Traceback (most recent call last):
  File "C:\Users\m\AppData\Local\Programs\Python\Python38-32\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\m\AppData\Local\Programs\Python\Python38-32\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "D:\Masaüstü\Projects\paper\venv\lib\site-packages\PyPaperBot\__main__.py", line 122, in <module>
    main()
  File "D:\Masaüstü\Projects\paper\venv\lib\site-packages\PyPaperBot\__main__.py", line 119, in main
    start(args.query, args.scholar_pages, dwn_dir, args.min_year , max_dwn, max_dwn_type , args.journal_filter, args.restrict, DOIs, args.scihub_mirror)
  File "D:\Masaüstü\Projects\paper\venv\lib\site-packages\PyPaperBot\__main__.py", line 45, in start
    downloadPapers(to_download, dwn_dir, num_limit, SciHub_URL)
  File "D:\Masaüstü\Projects\paper\venv\lib\site-packages\PyPaperBot\Downloader.py", line 63, in downloadPapers
    pdf_dir = getSaveDir(dwnl_dir, p.getFileName())
  File "D:\Masaüstü\Projects\paper\venv\lib\site-packages\PyPaperBot\Paper.py", line 31, in getFileName
    return re.sub('[^\w\-_\. ]', '_', self.title)+".pdf"
  File "C:\Users\m\AppData\Local\Programs\Python\Python38-32\lib\re.py", line 210, in sub
    return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object
ferru97 commented 1 month ago

Hi, sorry for the late reply, but PyPaperBot had been on standby until now. I've decided to resume development to improve it.

I've created a Telegram channel where you can suggest improvements, report bugs, or request custom data mining scripts. Feel free to join if you're interested with the following LINK

Thanks!