search

erikrose / peep

A "pip install" that is cryptographically guaranteed repeatable
MIT License
221 stars 28 forks source link

Avoid recompiling compiley packages #10

Closed erikrose closed 11 years ago

erikrose commented 11 years ago

It might be nice to avoid C recompiles. We could certainly avoid rebuilding a package that's already installed in the venv. We could assume that, if you used peep to install stuff in the venv in the first place, the hashes were verified. How would we go about making sure anything not in the requirements.txt any longer gets removed? Out of scope for peep? It would be pretty trivial to implement as a layer around pip freeze and pip uninstall, with a diff against requirements.txt: uninstall what shouldn't be there, then do the peep install.

rhelmer commented 11 years ago

So I notice that pip returns Requirement already satisfied (use --upgrade to upgrade) if any particular package+version already exists, while peep (testing 0.7 right now) always seems to reinstall everything, which is very slow, even for non-compiled code (obviously it's much worse for compiled code).

Is this the same issue? Sorry if it's not :S

erikrose commented 11 years ago

Yep, same issue. When you were using pip, how did you make sure packages that were no longer required got uninstalled?

erikrose commented 11 years ago

(He didn't.)

erikrose commented 11 years ago

We can call requirement.check_if_exists() before trying the download and either check its return value or requirement.satisfied_by afterward. I'm not sure which is correct.

Also, this uncovers a snake's nest around peep installing into non-empty venvs. What if somebody passes --upgrade to peep install? What would peep install --upgrade even mean? Does pip --upgrade even work or make sense with requirements files? Our abstraction is full of holes, and our hovercraft is full of eels. I suppose peep should really barf if it encounters any non-== requirement anyway, which solves the --upgrade problem.