Introduce `env` command to manage environments

[carstenbauer]

It would be nice to have a command for listing all environments. Currently the best one can do is gc, which will output a list at the top. Also, autocompletion for environment names when doing activate would be nice.

[00vareladavid]

At the very least: adding tab completion for pkg> activate --shared should be straightforward.

[carstenbauer]

It feels strange to me that julia manages environments in a somewhat hidden manner. It keeps a list of all active manifests (environments), as can be seen by gc, but doesn't let me see the list easily (this issue), doesn't have autocompletion (this issue), and also doesn't let me remove environments without deleting them on disk.

Why not make this more transparent?

activate could add an environment to the list, listenvs could list all known environments, and rmenv could remove an environment. The user would have more control over what julia tracks.

[fredrikekre]

It seems iffy that Pkg should be able to remove environments. What should be deleted? The Project.toml and Manifest.toml files?

[carstenbauer]

Nothing should actually be deleted. Julia should just stop keeping track of that particular environment (manifest). I.e. the manifest should be removed from the list that I see when I do gc.

[carstenbauer]

I have this use case in mind: My collaborator sends me a project folder. I wanna check it out so I do activate . and instantiate. However, from that point on I have the projects dependencies installed and can't free them up without deleting the whole project folder.

Why can't the user tell Julia to ignore this environment again? I mean it also didn't track it automatically when I created it on my disk. Only when I did activate it at some point.

[fredrikekre]

How about

(v1) pkg> listenv
[1] `/home/fredrik/.julia/environments/v0.7/Manifest.toml`
[2] `/home/fredrik/.julia/environments/v1.0/Manifest.toml`
[3] `/home/fredrik/TestProject/Manifest.toml`

(v1) pkg> activate 3

(TestProject) pkg>

[carstenbauer]

Looks great!

[carstenbauer]

It seems iffy that Pkg should be able to remove environments. What should be deleted? The Project.toml and Manifest.toml files?

Maybe, to avoid the confusion, the command to remove an environment from the list should be called untrack or similar rather than rmenv.

[StefanKarpinski]

What about an env command and an env list subcommand?

[carstenbauer]

What about an env command and an env list subcommand?

And env untrack? Sounds great to me.

Although I'd love to make a PR to get this merged soonish, I feel I'm not the right person for this. So hopefully someone else will give it a shot soon. I'd appreciate it!

[00vareladavid]

I renamed the issue to focus on env. #834 exists for tab completion on activate (which should be made a lot simpler by #839).

[carstenbauer]

Is someone working on this? I'd love to see this in 1.1 and I heard feature freeze is mid Nov.

[carstenbauer]

@fredrikekre talk at juliacon yesterday reminded me of this feature request. Someone from the audience basically wanted the env untrack functionality.

To add to the list of features that go under env, it would be cool to make stacking/unstacking environments (i.e modifying the load path) easier, such that one could use autocomplete for this as well. Something like env stack --shared SomeEnv. What do you think?

[Datseris]

I'm just adding an argument about why all of this is very useful. Just asked on Slack if there is some command exactly like env untrack described here. There isn't one yet.

Here is a reason why deleting the Manifest.toml or renaming it does not work. Scientists (at least in my workplace) are required to keep an archive of the code and data base they used in a scientific project, for several years after the project has been published. I have such bases, which are currently tracked by the package manager, thereby keeping installations of old packages (without much use). I do not want to (and legally, I also cannot) alter these archived code bases by removing or renaming any file, much less so such an important file as a Manifest.toml.

This is the currently best standing workaround by @fredrikekre

$ mv Project.toml _Project.toml
$ mv Manifest.toml _Manifest.toml
$ pkg gc
$ mv _Project.toml Project.toml
$ mv _Manifest.toml Manifest.toml

[carstenbauer]

FWIW, came up on slack again (someone was looking for env untrack functionality).

[KristofferC]

I think something like described here makes sense.

[carstenbauer]

Brought up once more: https://discourse.julialang.org/t/which-files-or-subdirectories-in-julia-can-i-delete/62230

[StefanKarpinski]

env untrack --old might work nicely with gc to allow reclaiming stuff that's rooted by environments that haven't been used in a long time. cf #2685

[halleysfifthinc]

I just bumped into needing/wanting this (I had 72Gb of artifacts) and making do with PkgCleanup from @giordano (removed 60Gb of artifacts after "untracking" old manifests). The default behavior of Pkg to never forget an manifest is somewhat inconsistent with the officially(?) blessed workflow recommendation to limit use of the default environment and use local env's for everything. How does that not lead to an enormous group of artifacts only kept around for old environments?