Pkg3: Environment manipulation in the language instead of with command line arguments

[tkelman]

If environment manipulation gets moved to a command-line invocation, people are probably going to want to run it most often from shell mode or via run(), which isn't any better than what we have now. Otherwise many Windows users aren't going to find it at all because the usual workflow there is often not via the command line. (IDE features can help, but shouldn't be required for this to be usable.)

As a thought, what if we make this tie into https://github.com/JuliaLang/julia/issues/17997 instead? Instead of loading packages into Main, load them into an environment-specific namespace. Enter and leave environments via Pkg3.enterenv("newenv"), Pkg3.leaveenv() etc. Make Pkg3.add, Pkg3.rm, and Pkg3.update take an environment argument. If the user attempts to update or remove a loaded package from an environment, that environment needs to be reset via something similar to workspace().

[StefanKarpinski]

That seems like a good idea although I'm a little skeptical about feasibility and to some extent usability. Can we have two completely unrelated environments in the same process at the same time? What if they use different versions of the same shared libraries? The workspace mechanism doesn't seem like it's been wildly successful in terms of usability and this seems a bit like that but more complex. Isn't independent processes a better, simpler, easier-to-implement model for this sort of thing?

[ChrisRackauckas]

The workspace mechanism doesn't seem like it's been wildly successful in terms of usability and this seems a bit like that but more complex. Isn't independent processes a better, simpler, easier-to-implement model for this sort of thing?

Juno has already gone the way of independent processes, implementing a process server so that way new processes open up instantly. This is something that seems to work very well, getting the "instant" response time while getting a truly cleared workspace. If the REPL acted the same that would be nice.

[tkelman]

You don't get any benefit from package management being an independent process if most users will want to run it from inside Julia most of the time. Upgrading happens often, removing occasionally. Both are safe if the packages they modify aren't currently loaded. They're only potentially problematic if the package is currently loaded. Rather than inconvenience everyone all of the time, I think it would be better to just warn on potentially problematic cases, like the loaded-in-memory version of a package no longer matching what's installed.

Environments might need to be separate processes, but they should be launchable and configurable from within the language. Anything you can't do from within the repl is going to rule it out from a large group of users.