petermr / pygetpapers

a Python version of getpapers
Apache License 2.0
78 stars 9 forks source link

biorxiv acting up strange #28

Closed ShweataNHegde closed 2 years ago

ShweataNHegde commented 3 years ago

Describe the bug I ran a usual query using --biorxiv API. In the recent version, it gives me errors described below.

To Reproduce INPUT:

C:\Users\shweata>pygetpapers --api biorxiv -k 10 -x --startdate 2021-01-01 -o biorxiv_test_5

OUTPUT:

INFO: Making Request to rxiv
Traceback (most recent call last):
  File "c:\users\shweata\appdata\local\programs\python\python38\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\users\shweata\appdata\local\programs\python\python38\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\shweata\AppData\Local\Programs\Python\Python38\Scripts\pygetpapers.exe\__main__.py", line 7, in <module>
  File "c:\users\shweata\appdata\local\programs\python\python38\lib\site-packages\pygetpapers\pygetpapers.py", line 602, in main
    callpygetpapers.handlecli()
  File "c:\users\shweata\appdata\local\programs\python\python38\lib\site-packages\pygetpapers\pygetpapers.py", line 588, in handlecli
    self.handle_query_download(args)
  File "c:\users\shweata\appdata\local\programs\python\python38\lib\site-packages\pygetpapers\pygetpapers.py", line 157, in handle_query_download
    self.rxiv.download_and_save_results(
  File "c:\users\shweata\appdata\local\programs\python\python38\lib\site-packages\pygetpapers\rxiv.py", line 223, in download_and_save_results
    result_dict = self.rxiv(
  File "c:\users\shweata\appdata\local\programs\python\python38\lib\site-packages\pygetpapers\rxiv.py", line 76, in rxiv
    ["doi"]] = total_papers_list[paper_counter]
IndexError: list index out of range

This is how it usually works.

petermr commented 2 years ago

I have run this on:

Welcome to Pygetpapers version 0.0.7.1. -h or --help for help

and cannot reproduce this:

Output

pygetpapers --api biorxiv -k 10 -x --startdate 2021-01-01 -o biorxiv_test_5
INFO: Making Request to rxiv
INFO: Making xml for paper
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:10<00:00,  1.26s/it]
INFO: Wrote metadata file for the query
INFO: Writing metadata file for the papers at /Users/pm286/workspace/pygetpapers/testing/biorxiv_test_5
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 2954.00it/s]
(base) pm286macbook:testing pm286$ pygetpapers --version
WARNING: Please specify a query
(base) pm286macbook:testing pm286$ pygetpapers
usage: pygetpapers [-h] [--config CONFIG] [-v] [-q QUERY] [-o OUTPUT] [--save_query] [-x] [-p] [-s] [-z] [--references REFERENCES] [-n] [--citations CITATIONS] [-l LOGLEVEL]
                   [-f LOGFILE] [-k LIMIT] [-r RESTART] [-u UPDATE] [--onlyquery] [-c] [--makehtml] [--synonym] [--startdate STARTDATE] [--enddate ENDDATE] [--terms TERMS]
                   [--api API] [--filter FILTER]

Welcome to Pygetpapers version 0.0.7.1. -h or --help for help

It created the output:

cd biorxiv_test_5/
(base) pm286macbook:biorxiv_test_5 pm286$ ls
10.1101_008326      10.1101_022459      10.1101_046052      10.1101_067736      rxiv-results.json
10.1101_010553      10.1101_035972      10.1101_060012      10.1101_078378
(base) pm286macbook:biorxiv_test_5 pm286$