Better failure message when a project isn't found on PyPI

[edmorley]

Running latest master (4600247535d752b2728b876cacf01b914bd414cd) of caniusepython3 against this requirements file, results in:

(caniusepython3) [~/src/treeherder]$ caniusepython3 -r requirements/common.txt
Finding and checking dependencies ...
[WARNING] Stale overrides: set([u'regex', u'certifi', u'testresources', u'ipaddress', u'futures', u'bcdoc', u'mox3', u'gevent', u'requests-oauthlib', u'httpretty', u'pyvirtualdisplay', u'logbook', u'pyodbc', u'numpy', u'extras'])
[ERROR] Failed to get external data for https://www.red-dove.com/pypi/projects/B/Brotli/project.json: HTTP Error 404: Not Found
Traceback (most recent call last):
  File "/home/Ed/.virtualenvs/caniusepython3/lib/python2.7/site-packages/distlib/util.py", line 764, in _get_external_data
    resp = urlopen(url)
  File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 475, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 558, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 404: Not Found
[WARNING] Brotli not found

You need 2 projects to transition to Python 3.
Of those 2 projects, 2 have no direct dependencies blocking their transition:

  datasource
  mozlog

Ideally there would be:

• More explanation about what stale overrides are (presuming it's a prompt to file an issue against this repo) - ideally the warning should make that clear.
• No errors.

The Brotli requirement is specified as: https://github.com/google/brotli/archive/v0.3.0.zip#egg=Brotli==0.3.0.

[brettcannon]

Yes, ideally there would be no errors, but this is software so that's not about to happen. ;)

I'll open a separate issue to more clearly explain what a stale override even means, but the Brotli failure is an upstream problem with distlib missing data.

[edmorley]

Yes, ideally there would be no errors, but this is software so that's not about to happen. ;)

Of course; imprecise choice of words on my part :-)

Many thanks for filing #132 and for commenting about Brotli on the distlib ticket.

[brettcannon]

Actually, now I see the issue: the brotli package you are pulling in is not hosted on PyPI/pypi.io and so distlib doesn't know anything about it. So this isn't a bug in caniusepython3 as it only operates on projects hosted on PyPI.

A better error message pointing out that the failure may be due to the project not being on PyPI is reasonable, though.