search

PolicyStat / terrarium

Package and ship relocatable python virtualenvs, like a boss.
https://terrarium.readthedocs.io/en/latest
BSD 3-Clause "New" or "Revised" License
171 stars 14 forks source link

Symlinks and relative paths in easy_install.pth break portability of virtualenvs #20

Open christian-oudard opened 11 years ago

christian-oudard commented 11 years ago

So, in our server setup, we have numerous symlinks to the various code versions. We are running the terrarium install from a symlinked "~/current/" directory, and it creates a frozen virtualenv with a non portable easy_install.pth. It contains entries like this:

../../../../../../../mnt/srv/server_name/releases/20130116140210-heads-master-0-g47d7e44/src/pymysql

When it should actually contain: ../../src/pymysql

This is basically the relative path going all the way up to the root, then back down the non-symlinked path to the same place as we started.

The problem here is that we end up including "server_name" in the .pth file, which breaks when we are on a different game server installing the same packages, and the game name is different.

I see three possible solutions here: 1) Always have server_name be the same across games, which is confusing and would take a lot of work. 2) Always make sure that you never run terrarium from a path that contains symlinks. This is something that should be well documented by terrarium. 3) Make terrarium generate easy_install.pth more intelligently.

Can we try to figure out how to do option 3?

winhamwr commented 11 years ago

My guess is that the symlinked directory use-case is fairly common. Versioning our virtualenv on deployment, just like we do with our code, is actually on my todo list.

It definitely seems like option 3 is doable.

christian-oudard commented 11 years ago

So, I've fixed it with option 3, but it required forking virtualenv: https://github.com/pypa/virtualenv/pull/389

Not sure if it's possible to do something like this for older versions of virtualenv, just by tweaking it from the terrarium side.

kylegibson commented 11 years ago

terrarium can definitely post-process the easy_install.pth file for older versions of virtualenv

christian-oudard commented 11 years ago

Cool. I'm not really sure how to approach that though. Maybe we could just re-implement make_virtualenv_relocatable on our own?

kylegibson commented 11 years ago

That may do it. It will be awhile before I will have the time to tackle this, so feel free to take a stab at it meanwhile.

christian-oudard commented 11 years ago

I think I got it. I put up a pull request, but I'm not sure how to link it to this ticket.

kylegibson commented 11 years ago

Christian, if you want, you can use hub to turn this issue into a pull-request. Something like this:

hub pull-request -i 20 -b PolicyStat/terrarium:master -h brooklynpacket/terrarium:master