search

conda-forge / miniforge

A conda-forge distribution.
https://conda-forge.org/miniforge
Other
6.51k stars 331 forks source link

Don't add to Path by default? #453

Open mdhaber opened 1 year ago

mdhaber commented 1 year ago

Comment:

On Windows (and perhaps other platforms), the default behavior of the mambaforge installer is not to add the directory containing mamba to the Path environment variable.

Is it recommended to use mamba only via the Miniforge Prompt (or use the full path to mamba from other shells)?

What conflicts are expected if this directory is added to the path?

hmaarrfk commented 1 year ago

I don't use windows often, but when I tried things last week, I was surprised by this (new to me) behavior. @jaimergp maybe the default flags got changed and we missed the new variable?

jaimergp commented 1 year ago

Let me check!

Maybe we missed a syntax or added a typo that prevents the behavior, but the change was definitely was not intentional. We have commented on this issue at https://github.com/conda-forge/miniforge/issues/421 too, btw.

jaimergp commented 1 year ago

Ah wait, @hmaarrfk this change would explain the behaviour we are observing. Maybe it's time to revert?

hmaarrfk commented 1 year ago

Ah wait, @hmaarrfk this change would explain the behaviour we are observing. Maybe it's time to revert?

This change is trying to attempt the exact problem we are seeing right? it seems like we want it in......

https://github.com/conda/constructor/commit/a1efe3b01bdbbcfde42d51bc6677cf23290ccb35#diff-af1b4db4a14299e7d6a5781c05afe756f88a088a81fbc74f97932ac467d3f821R617 image

indicates that the flags we have now should have the intended effect

jaimergp commented 1 year ago

What's the intended effect?

My understanding right now is that some users expected mamba to be available in PATH, but initialize_by_default is being set to False on our construct.yaml, which leaves the "Add To Path" unchecked on Windows. Users would need to manually check that box for the changes in PATH to happen.

Note that initialize_by_default=False is desired on SH installers, but not on PKG/EXEs. See the docs for this key (text predates my time but I think the intent there matches users expectations).

hmaarrfk commented 1 year ago

oh. I think i confused the "Regsiter default Python" with "Add to path"

image

The add to path checkbox is indeed not checked.

My apologies.

My understanding is that on windows, the path is so global, that adding the conda installer is nearly guaranteed to break your windows installation since newer dlls will now be in standard program paths.

On linux, the "activation" is done in interactive shells, which doesn't change how other programs not started from the terminal behave.

jaimergp commented 1 year ago

I agree it's a bad idea and I wouldn't want that added back. If only it was just /condabin, but it actually adds all the bin paths in base... That's almost guaranteed to interfere with other things in the system; if not at install time, at some point when a problematic package is added to base.

But it's true that it was the default behavior not too many releases ago, so some users might be expecting it. We should at least document the change and rationale in a pinned issue or something similar?

mdhaber commented 1 year ago

Is it possible for a subset of the executables, like mamba, to be added to the path? I don't imagine there would be conflicts with that, right?

If not, is it recommended to use mamba only via the Miniforge Prompt (or use the full path to mamba from other shells)?

jaimergp commented 1 year ago

$PREFIX/condabin exists for that purpose, but adding support for that would require a new option to maintain backwards compatibility. And yes, the recommended way on Windows is usually via the custom shortcut anyway. Otherwise the shell activation logic doesn't work.

isuruf commented 1 year ago

Otherwise the shell activation logic doesn't work.

It's not a technical restriction though. conda init only adds condabin to PATH, but does not do auto activate base regardless of the auto_activate_base option.

jaimergp commented 1 year ago

On Powershell it also writes to some profile files. For CMD it does some registry manipulation... Maybe not too critical, but still some side effects some people want to avoud.

That said, constructor doesn't run conda init on Windows as part of the Add to PATH stuff, it literally adds some paths to PATH.

Maybe it should, but currently it doesn't 😬

hmaarrfk commented 1 year ago

If not, is it recommended to use mamba only via the Miniforge Prompt (or use the full path to mamba from other shells)?

I've had the habbit of when i want to use mamba/conda to just open the miniforge prompt.

tobiasdiez commented 1 year ago

As a related issue, the winget installer uses the AddToPath switch (https://github.com/microsoft/winget-pkgs/blob/91726e8f822b864c1ce071521f1a9a49217c88a3/manifests/c/CondaForge/Mambaforge/23.1.0-1/CondaForge.Mambaforge.installer.yaml#L19). This should be aligned with whatever is the recommended default. It also doesn't seem to work currently, at least nothing was added to path in my case.

kfsone commented 1 year ago

The "Install for All Users" option currently does NOT allow for adding anything to the path, the net result of which is that none of the documentation works out of the box.

Fair enough, you made a selection, but this is not a typical consequence of making that selection in a Windows app and neither the installer nor the documentation call this out.

Windows has two levels of environment variable, user and system, and "$PATH" is the combination of both for any given user - just a formalized implementation of having /etc/shrc and ~/.shrc with both setting PATHs.

Typical conventions I've seen Windows installers use for scenarios like this are:

Meanwhile, going back to the per-user option, so I can have it added to my path by checking a box.

kfsone commented 1 year ago

The per-user install actually already has a red caution: image

This caution seems weirdly at odds with my understanding of how Windows users operate, and it sounds like a misunderstanding from a non-windows user who sometimes uses the MSBuild tools or Visual Studio (not code), and thinks that you have to open a tweaked console to access the command line tools, rather than that there's a backup mechanism for accessing them on build machines etc.

Not having it in the path is hyper-inconvenient if you are working on a Windows box directly or remoting in VS code and want to use the terminal; if you are sshing/ps-remoting into a machine... Or in just about every normal use case?

BartoszPiechnik commented 1 year ago

Not having it in the path is hyper-inconvenient if you are working on a Windows box directly or remoting in VS code and want to use the terminal; if you are sshing/ps-remoting into a machine... Or in just about every normal use case?

I totally agree with pretty much everything @kfsone said.

I just installed mamba on a fresh Windows installation, so just to give you perspective how it looks from a Windows user.

  1. You download the installer from conda-forge.
  2. Install it using the RECOMMENDED settings.
  3. Try the first, most basic command from the documentation.

AND IT DOESN'T WORK.

I know why it doesn't work. 'mamba' is not in the PATH. Common practice in this case is to add new software to the PATH. BUT, this is not recommended. And there is NO suggestions in mamba documentation on how to make it work on Windows (if common practice solution is not recommended).

I don't want to get involved here in the technical discussion, just wanted to share an honest feedback, fully from user perspective.

I guess the minimal solution would be to add to the documentation some guidance on how to make mamba usable from PowerShell/CMD if adding to PATH is the recommended way.

hmaarrfk commented 1 year ago

miniforge maintainer gere

Windows users operate

fundamentally, miniforge, and in my opinion, conda is not designed for the average Windows users.

it is designed for developers who wish to tweak and adjust packages and dependencies.

conda-forge has a tendency to install bleeding edge packages, which would often break other software. libraries often clash between other software and conda.

this we must be super careful in how libraries are chosen. we want that the conda and mamba executable's libraries are chosen from the environment, and that others use the standard windows executable. this is really hard to do by "just add things to the path"

I would argue that windows users expect a nice menu icon for every application they install. they expect applications to not step on each other's toes.

this is achieved in my mind with the default installation. you click next a few times, then you get a nice icon in the start menu that says "Miniforge 3 Prompt". and you click on that and the mamba and conda command work.

as for "all users", I've never used it before. generally, that seems dangerous for rapidly changing software like the libraries released on conda-forge.

mdhaber commented 1 year ago

this is achieved in my mind with the default installation. you click next a few times, then you get a nice icon in the start menu that says "Miniforge 3 Prompt". and you click on that and the mamba and conda command work.

@hmaarrfk I found this eventually, but I'd suggest documenting this at https://github.com/conda-forge/miniforge. Actually, I knew to look for a prompt based on experience with conda, but even then I was looking for a "mambaforge" prompt, and that didn't exist.

it is designed for developers who wish to tweak and adjust packages and dependencies.

This may be true; I develop SciPy. But because of that, I have a lot of other software that is also easier to use in its own prompt (e.g. mingw/MSYS2, git bash). So if it is possible to add a subset of miniforge software to the prompt so it could be used in other prompts, that would be helpful.

hmaarrfk commented 1 year ago

"mambaforge" prompt, and that didn't exist.

this is actually a bug and rather unfortunate. we have been trying to look into it with little success. given that miniforge3 and mamba forge have been (very) recently made identical, I am much less inclined to work to address this.

PRs welcome to add documentation and screenshots here that may help get users going. we have some similar instructions for one line copy paste installers for Linux users.

mdhaber commented 1 year ago

PRs welcome to add documentation and screenshots

I don't see any usage instructions that I can add to - just install and uninstall. Is that right?

In that case, I would suggest that I could:

When miniforge/mambaforge is on the path (default on Mac and Linux), its versions of the conda and mamba commands can be used at any command prompt. The most notable difference is that the default channel for packages will be conda-forge.

Is this accurate? Is there more to be said?

On Windows, miniforge/mambaforge is not added to the system path by default, so conda/mamba cannot be used from ordinary command prompts without including the full path of the executables. Instead, it is recommended to use the Mambaforge Prompt, available from the start menu. If desired, the ___ folder may be added to the path environment variable manually.

I don't have Mambaforge installed on Windows right now, so please correct me if "Mambaforge Prompt" should be stylized differently. Also, what folder should be added to the path if the user wants to take the risk?

hmaarrfk commented 1 year ago

I don't see any usage instructions that I can add to - just install and uninstall. Is that right?

The README.md is as elaborate as our instructions go for now. Feel free to add to there. You can add a directory for the screenshots. Sometimes assets is used.

The "Windows" installation section is rather short: https://github.com/conda-forge/miniforge#windows it definitely needs your help.

Rename "Local usage"

If this is referring to https://github.com/conda-forge/miniforge#local-usage then this was originally meant to be "Local usage of the repository that builds a Miniforge3.sh file". Again, PRs that help with new wording are appreciated.

As for the documentation, lets focus on Miniforge3 I'm proposing we deprecate the mentions of Mambaforge in order to streamline instructions now that they have been made identical: https://github.com/conda-forge/miniforge/pull/488

mdhaber commented 1 year ago

I added a bit that would have helped me in gh-453.

Separate from the documentation, is it possible to change the bit in the installer: image

Neither "Command Prompt" nor "Powershell" menus are added to the Windows Start Menu. An app called "Miniforge Prompt" is added to the start menu, so I think it should read:

Instead, use the "Miniforge Prompt" added to the Windows Start Menu to invoke the command line software.

or, more specifically, "...to use conda/mamba, if those are indeed the primary features of Miniconda. (That's all I'm aware of.)

tobiasdiez commented 1 year ago

wouldn't it be safe to add at least the commands conda and mamba to the path? This shouldn't lead to much issues, even if say anaconda is installed (then its only a question which conda is invoked). VsCode, for example, is doing something similar by adding <vscode install path>/bin to the path and having a small code.cmd in that folder that invokes the executable.

Alternatively, the docs should have at least contain a pointer to set-alias so that one can add aliases in powershell for conda and mamba.

hmaarrfk commented 1 year ago

wouldn't it be safe to add at least the commands conda and mamba to the path?

I am not sure there is a direct analog to this on Linux. Do you know of a specific mechanism on windows that allows this? Windows seems to suggest using "App Paths" https://learn.microsoft.com/en-us/windows/win32/shell/app-registration#using-the-app-paths-subkey

VsCode, for example, is doing something similar by adding /bin to the path and having a small code.cmd in that folder that invokes the executable.

This kind of change would be much larger for conda environments. Presently, conda and mamba are just "standard" executables installed in the "standard" way. Their folder contains many more executables

Alternatively, the docs should

It would really be more appreciated if this were rephrased as "Alternatively, would be OK if I made suggested to the docs that....".

To get back to the original question:

Is it recommended to use mamba only via the Miniforge Prompt (or use the full path to mamba from other shells)?

Yes, the recommended way to use conda and mamba on windows is to use the provided Miniforge Prompt.

tobiasdiez commented 1 year ago

wouldn't it be safe to add at least the commands conda and mamba to the path?

I am not sure there is a direct analog to this on Linux. Do you know of a specific mechanism on windows that allows this? Windows seems to suggest using "App Paths" https://learn.microsoft.com/en-us/windows/win32/shell/app-registration#using-the-app-paths-subkey

VsCode, for example, is doing something similar by adding /bin to the path and having a small code.cmd in that folder that invokes the executable.

This kind of change would be much larger for conda environments. Presently, conda and mamba are just "standard" executables installed in the "standard" way. Their folder contains many more executables

It's the same for vscode. The main directory contains quite a few other exes that shouldn't end up on the path. The idea is that you create a subdirectory that only contains the cmd files that you want on the path (and that then simply forward to the exe in the parent dir). So very similar to the current condabin folder.

hmaarrfk commented 1 year ago

from skimming app paths documentation quickly, it seems like the right solution.

can you find a reason why it might not be?

tobiasdiez commented 1 year ago

This only works for Start > Run, and is not active in the command line nor the powershell console.

hmaarrfk commented 1 year ago

So looking at my Miniforge installation on windows, it seems that there is already a directory

C:\Users\username\miniconda3\condabin

Looking through constructor's code, it seems that eventually this python function is called: https://github.com/conda/constructor/blob/8e65a5df74a5b53cd7c4e59dc062f2d9b66e8dc8/constructor/nsis/_nsis.py#L265

It seems that this particular location condabin is not added. Maybe it should be, and maybe the other locations should be removed.

The steps would be:

If manually adding condabin to the path helps you, I would consider documenting it. But I think it is hard to confirm without testing that it does.

mdhaber commented 1 year ago

That's actually in the readme as of gh-453. (Looks like I need to go back and fix the link rendering, though.)

hmaarrfk commented 1 year ago

can we also avoid the link to geeksforgeeks i really don't like that website for documentation. I often find it lacking.

mdhaber commented 1 year ago

Sure; please suggest an alternative.

hmaarrfk commented 1 year ago

https://learn.microsoft.com/en-us/previous-versions/office/developer/sharepoint-2010/ee537574(v=office.14)#to-add-a-path-to-the-path-environment-variable

bjarthur commented 8 months ago

the problem with the miniforge prompt is than it is a command prompt not powershell. any chance we can get the latter? coming from linux/macos there is much less cognitive burden for me to use powershell as the commands more closely resemble bash.

hmaarrfk commented 8 months ago

@bjarthur maybe the discussion in https://github.com/conda-forge/miniforge/issues/516 is of use to you.

bjarthur commented 8 months ago

thanks @hmaarrfk ! i ran conda init powershell in the miniforge prompt as instructed in #516. now i can open a regular (ie independent from miniforge) powershell prompt and conda is in the path. does that mean stuff is going to break as you mentioned earlier?

My understanding is that on windows, the path is so global, that adding the conda installer is nearly guaranteed to break your windows installation since newer dlls will now be in standard program paths.

hmaarrfk commented 8 months ago

i believe the init commands are designed to "add things" dynamically when a shell starts up.

For example, .bashrc is only used for "interactive shells", so users that run batch jobs on servers are often surprised when they can't access their conda environments.

I'm not so sure about the powershell specifics, but it is my understanding that the windows registry, and the PATH as you would add it through the control panel are global "at startup" settings. conda init (in my mind) may be designed to avoid these "global settings".

kfsone commented 7 months ago

Nobody is going to expect a miniforge icon on their desktop - Ubuntu, Mint, Rocky, Zorin, Arch, Gentoo, Mac, Win, WSL...

It's a command-line application.

When you install Git, Python, Ruby, Perl, Mercurial, Cargo, Golang, NeoVim, VsCode, PyCharm, CLion, ... on Windows, via package manager or installer, they all add themselves to the path, or give you a script to do it or a pasteable fragment to do it, and/or they give you the option to enable/disable that via a checkbox. But they assume you're going to be operating them from the command line, or using GUI tools that will expect to find them in the path.

The union of miniforge <-> windows users is going to be primarily cross- or multi-platform, cli-centric people. There are a number of disciplines for which Windows has become an ideal base from which to work on the other platforms. Hell, for me, after 30 years as a sh/ksh/csh/bash power-user doing awk/sed/grep/tr/cut command-line golf as often as breathing, the transition to powershell was directly comparable to going from Perl to Python: "how am I supposed to split a string if there's no split keyword?" -> "strings are objects, use the method"; "how am I supposed to do a tr on the output of a command if the output isn't text?" -> "it is text, but it's a string, use the .replace method instead of piping to another process"...

Not trying to sell or convert, only to dismiss the idea that Windows is some distant, unrelated 3rd party that no developer or command-line operator would use except for Active Directory and Exchange administrators and grandma using the re-gift her 2nd cousin got instead of a PS5.

If you want a better sense of who is ... working from windows? ... take a look at who contributors are to PowerShell, windows terminal, and/or which msers are contributing to docker/linux.

ncoghlan commented 6 months ago

I happened to be thinking about this issue while installing Git for Windows, and noticed that their installer had a dedicated page specifically relating to the PATH config, rather than combining it with other optional behaviours: image

bilderbuchi commented 5 months ago

The way the Git for Windows installer handles this seems most reasonable to me as a user.

FWIW, as a longtime mini/conda/mambaforge user on Windows, the setting about adding to PATH is the only thing I always check (despite the warning) to get the convenience of not having to use a custom, separate terminal just for Python work. I have not yet observed anything breaking, so far.

jaimergp commented 5 months ago

One thing to consider is whether we change constructor to add condabin to PATH, but not the full bin / Scripts directory.

BrenBarn commented 1 month ago

This situation is rather confusing for Windows users. I agree with others that the best solution seems to be to find a way to include just the conda/mamba command itself on the path, without also including a bunch of DLLs that will mess up ordinary Windows use.