ncbi / pgap

NCBI Prokaryotic Genome Annotation Pipeline
Other
310 stars 90 forks source link

HTTP Error 403: rate limit exceeded #138

Closed ynanyam closed 3 years ago

ynanyam commented 3 years ago

We are experiencing HTTP Error 403 issues which is preventing us from using PGAP.

Even simple commands like pgap.py --list return that error.

Any possible issues with API limits imposed by Github? --no-internet and --no-self-update doesn't seem to help either.

$ pgap.py --list -d
Traceback (most recent call last):
  File "pgap.py", line 838, in <module>
    main()
  File "pgap.py", line 804, in main
    params = Setup(args)
  File "pgap.py", line 446, in __init__
    self.remote_versions = self.get_remote_versions()
  File "pgap.py", line 510, in get_remote_versions
    response = urlopen('https://api.github.com/repos/ncbi/pgap/releases/latest')
  File "/software/7/apps/python_3/3.6.6/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/software/7/apps/python_3/3.6.6/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/software/7/apps/python_3/3.6.6/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/software/7/apps/python_3/3.6.6/lib/python3.6/urllib/request.py", line 570, in error
    return self._call_chain(*args)
  File "/software/7/apps/python_3/3.6.6/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/software/7/apps/python_3/3.6.6/lib/python3.6/urllib/request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: rate limit exceeded

-- Yash

whlavina commented 3 years ago

Thank you for the report. Indeed, the code should respect rate limits on API calls, and should avoid those calls altogether if requested to avoid internet access. This should be a reasonably easy fix; we'll work on it.

whlavina commented 3 years ago

We have implemented a fix, which should make it into the next release. I am closing this issue. Thanks for your feedback!