Managing multiple environments

[majosm]

I'm trying to figure out how to use emirge (possibly in conjunction with other tools, e.g., conda) to set up and quickly switch between multiple mirgecom toolchains.

As a concrete example, suppose I have one feature that depends on a post-array-context toolchain and another that is still pre-array-context. Previously, when switching between tasks I would have to go into the individual packages' directories and manually switch branches. I was hoping I could avoid this by switching over to emirge. With the old submodule approach I had a mental model for how this would work, but I'm not sure how to do this with requirements.txt. Any tips? (Is this documented somewhere? I couldn't find it.)

[lukeolson]

Can you use two different environments for this?

• Create env1, checkout feature 1, install.
• New terminal. Create env2, checkout feature 2, install.

[majosm]

I think(?) I would have to create a separate clone for each feature in order to make that work (since the sub-packages are downloaded to the main emirge directory and installed with install -e).

I could do that... kinda hoping there's a cleaner way though. 🙂

[lukeolson]

the main emirge directory and installed with install -e

Ah. Installing in development mode might not work with this.

[inducer]

@majosm I think the multiple checkouts route is the only way that works right now.

We coud hack https://github.com/illinois-ceesd/emirge/blob/master/install-pip-dependencies.sh so that it doesn't just blindly clone repos, but instead is able to switch branches in an existing environment (based on requirements.txt in mirgecom). Maybe @matthiasdiener can comment once he's back.

[MTCam]

TEESD needs to have multiple environments just like you describe @majosm. This install command is currently working on emirge@Option--env-name on Lassen:

./install.sh --conda-prefix=$CEESD/conda-install --env-name=test.env1 --install-prefix=$CEESD/mirgecom-install-1 --branch=mybranch

This command will install a working environment for mirgecom@mybranch in the directory $CEESD/mirgecom-install-1 and this environment has its own conda env that you can load to work on that branch. Any number of these can be installed side-by-side, and multiple installs can share conda envs if you want.

When we do continuous builds (if we ever do), TEESD will be doing multiple builds at once, which is why it needs independent environments. These changes are in PR now. #65 #60

Does this help in your situation?

[MTCam]

fyi my environment setting on Lassen so far pre-emirge was whatever the default is +

module load gcc
CEESD=/usr/workspace/wsa/xpacc/*/CEESD/Lassen
CC=mpicc
CXX=mpicxx
FC=mpif90

[majosm]

I'm not really loving the idea of having separate local subpackage git repos for every feature branch. It means you have to do the initial git setup work (adding remotes, etc.) every time you create a new branch. And if you want to do something that operates between branches (merge, cherry-pick, diff, ...), you can't directly; you have to use workarounds like adding a remote for the other local git repo or pushing it to public first. Seems awkward.

[MTCam]

I'm not really loving the idea of having separate local subpackage git repos for every feature branch. It means you have to do the initial git setup work (adding remotes, etc.) every time you create a new branch. And if you want to do something that operates between branches (merge, cherry-pick, diff, ...), you can't directly; you have to use workarounds like adding a remote for the other local git repo or pushing it to public first. Seems awkward.

I don't understand what you mean here. I develop on multiple branches, each requiring a different set of sub-packages (e.g. meshmode, grudge) - and I do not face any of these troubles. Can you explain?

[majosm]

So, with your current setup you're creating separate installs for different features, right?

Suppose you start with one feature and install with --install-prefix=$CEESD/feature1 --branch=featurebranch1. You do some work on (e.g.) mirgecom in there -- you make some commits, set up some remotes (your own GitHub, maybe those of people you're collaborating with). So far so good. Now you want to start working on a second feature, so you create another install with --install-prefix=$CEESD/feature2 --branch=featurebranch2. The mirgecom repo in this install doesn't have the remotes you created in the other one (only origin), so you have to set those up again. Now you do some work on feature2. Suppose at some point you want to run a git command that operates between the two features (maybe you want to temporarily borrow a fix via cherry-pick, or diff something between them). You can't do this directly, since featurebranch1 is in one mirgecom repo and featurebranch2 is in the other. Instead of, say, git diff featurebranch1 featurebranch2, you'd have to do something like

git remote add feature1 /path/to/feature1/install/mirgecom
git fetch feature1
git diff feature1/featurebranch1 featurebranch2

or

cd /path/to/feature1/install/mirgecom
git push origin featurebranch1
cd /path/to/feature2/install/mirgecom
git fetch origin
git diff origin/featurebranch1 featurebranch2

both of which are kind of yucky workarounds, IMO.

[MTCam]

Thanks for that detailed explanation @majosm. I didn't follow before because adding other remotes and stuff is not something I actually do (often). I'm always doing the clunky thing of checking out the different feature when I want to work on it as you describe. I can't really imagine another way to do that.

The new options do appear to address some of the issues you mentioned at the top of this thread:

The (now merged #60) fixes the issue where all of mirgecom and its dependencies are installed under the emirge tree (i.e. if you use the --install-prefix option), and #65 creates (or uses) the specified environment for each install (iff you use the --env-name option). Otherwise, it does the same thing it did before the options were added.

It seems that some additional tools would be needed to better manage the situations where inter-feature development is desired.

[matthiasdiener]

Just to add a detail - since we are supposed to use forks for pushing PRs to other repos (loopy etc.), we won't be able to avoid working with multiple remotes for the repos.

[MTCam]

So far, I've been able to manage this by putting my personal repo as the source in the requirements.txt file:

git+https://github.com/MTCam/meshmode.git

Things I push there automatically update the PR.