erikrose / peep

A "pip install" that is cryptographically guaranteed repeatable
MIT License
221 stars 28 forks source link

Support offline use #14

Open erikrose opened 11 years ago

erikrose commented 11 years ago

Another use case, besides trustability, for a local PyPI mirror is resilience to network failures. It's nice to be able to deploy even when PyPI is down or inaccessible. At the moment, peep installing with --download-cache still insists on hitting PyPI, failing out if it can't. That seems unnecessary. Find the right set of flags to pass to pip to make that not happen, or at least understand why it needs to happen.

erikrose commented 11 years ago

buildout pulls it off: https://pypi.python.org/pypi/zc.buildout/1.5.2#installing-solely-from-a-download-cache. In the worst case, we can implement out own trivial requirement-to-tarball-name mapper.

erikrose commented 11 years ago

Pip can do it, too: http://www.pip-installer.org/en/latest/cookbook.html#fast-local-installs

But when you replace "pip" with "peep" in the second code snippet, this happens:

(smoo)[18:10:07 ~/Checkouts/peep]% peep install --no-index --find-links=/Users/erose/.pip/download -r /Users/erose/Checkouts/socorro/requirements/dev.txt
Ignoring indexes: https://pypi.python.org/simple/
Downloading/unpacking sqlalchemy-citext==1.0-2
  Running setup.py egg_info for package sqlalchemy-citext
    Traceback (most recent call last):
      File "<string>", line 16, in <module>
    IOError: [Errno 2] No such file or directory: '/Users/erose/Virtualenvs/smoo/build/sqlalchemy-citext/setup.py'
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 16, in <module>

IOError: [Errno 2] No such file or directory: '/Users/erose/Virtualenvs/smoo/build/sqlalchemy-citext/setup.py'

----------------------------------------
Command python setup.py egg_info failed with error code 1 in /Users/erose/Virtualenvs/smoo/build/sqlalchemy-citext
Storing complete log in /Users/erose/.pip/pip.log

I'm sure it's to do with our blithely rewriting commandlines to make multiple internal peep calls. We should be able to work this into a solution; all the parts are there.

erikrose commented 11 years ago

The other thing to consider is that the format of a pip download cache is not the same as what's expected by --find-links. --download-cache emits filenames like https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fp%2Fpyelasticsearch%2Fpyelasticsearch-0.3.tar.gz, while --find-links needs things like pyelasticsearch-0.3.tar.gz.