Add menu to change console environments

To make it easier to switch what a new console defaults to and to change the environment for an existing console, we can add a menu from the lower status bar similar to the language server to its left.

Here is my proposal:

The default option will always be at the top, separated from the other options, and has the (Default) label. All other options should be listed below alphabetically with a check (Material Icons check) to the left of whichever option is being used by the currently selected console.

To change the default, hovering over that option opens the secondary menu. It follows the same pattern of alphabetic listing and check for currently selected default/

What if users don't venture into using envs and just use the base env for all their computations? In that case, consoles would be named base, base (1), etc, which I think it'd be really confusing for newbies. They could ask: why consoles are not named Console 1, Console 2, etc (as before) and instead they have this weird base name?

This is kind of the same scenario as the above one, but for our installers, i.e. consoles would be named default, default (2), etc, which I also find confusing for newbies and people not familiar with envs.

Okay, so in that case, we could just go with the Python X.Y[.Z] name for all consoles that aren't started in an explicit user-named including base and the Spyder internal environment. We could then show the full details (name, path, etc) in a tooltip for disambiguation and users interested in that, as I believe Isabela previously suggested.

And what if two non-env installations have the same Python version? For instance, two Homebrew and MacPorts Pythons? In that case the Python version is not enough, unless we have Python X.Y.Z (2), Python X.Y.Z (3), etc, to be consistent.

That's exactly what I was proposing. Another alternative instead of Python versions, since most Pythons are either the system Python, Python.org or come with known distributions like MacPorts, Homebrew, Anaconda/Miniconda/Miniforge/Mambaforge, WinPython, deadsnakes etc. (i.e. most of them) where a parent dir in the path we can detect can be matched to a known distribution, we could give the console(s) that name instead. And for the Spyder-managed default env, we don't need to use whatever the actual name is; we can give it a clearer/more user friendly name instead—just Spyder, for instance. Or even the old Console.

If we didn't put the console number in parenthesis, we could put the Python version number in parenthesis, providing nearly complete disambiguation, as it is extremely unlikely users would ever have two separate installs, by the same method (if that's even possible), and with the same Python version. So, my revised suggestion is as follows

<Installation/Environment> [- <Special-type>] (Py<X.Y>) [<Number if >=2>]

I'm not even sure we need the <Special-type> in the tab name; we could have a special tab icon instead (Sympy, Cython or Pylab [MPL] all have circular-ish icons) which would take up less space, be less obtrusive and be easier to spot at a glance. This would simplify the above down to:

<Install/Env> (Py<X.Y>) [<N>]

So for example:

Anaconda base: Anaconda (Py3.7), Anaconda (Py3.7) 2, Anaconda (Py3.7) 3 Miniconda environment: pandas-env (Py3.10), pandas-env (Py3.10) 2 Homebrew: Homebrew (Py3.9), Homebrew (Py3.9) 2 System Python: system (Py3.8), system (Py3.8) 2 Spyder default: Spyder (Py3.9), Spyder (Py3.10)

For the Python.org Python or custom cases where we can't determine a specific distribution name, environment name or system Python, we could do Console (Py[X.Y]) [<N>], i.e. the same pattern as the above, or Python X.Y.Z [<N>].

What I'd like to highlight with my previous comments is that you (i.e. both @CAM-Gerlach and @mrclary) seem to be optimizing the naming convention for envs.

I've now modified the convention substially in my latest proposal above to provide a user-friendly name for a variety of env and non-env Python environments.

And I think using envs is still an advanced feature that perhaps not many people rely on.

Well, they really shouldn't be, because as we all know that is how users routinely break things. There has been a concerted push recently to encourage users to always use envs and make it as easy as possible to do so, with e.g. PEP 668 and PEP 704 on the pip side to prevent pip from installing if an environment isn't activated, conda/conda#12245 to do the same for base on the Conda side, Anaconda Navigator and of course own own work on the Conda-based installers and the environment management plugin in Spyder.

However, even for users not using environments but just specific Python install(s) like you mentioned above, then the revised solution here is equally helpful if not more so, as explained below.

So, I propose that we keep using Console X when the default interpreter is selected in Preferences. In that way, consoles attached to envs would be treated as our other special consoles (Pylab, Sympy, dedicated, etc) by having a different name. And I guess users should have no problem to get familiar with them due to that.

If "Python X.Y" is not sufficiently unique and the user may still not know from which Python the console is being launched from (Homebrew, MacPorts, system Python, Anaconda, etc), then "Console" is even less unique and gives the user even less information on which to go on; my revised proposal aims to specifically address your feedback and tell the user exactly what Python installation they are running right in the Console tab. I'm sure you've seen many users getting confused about where the Python they're using is coming from and where they can install things to it, so this should be a big help in that regard.

I think having three additional submenus for Pylab, Sympy and Cython inside the Special consoles submenu is fine.

Yeah I don't really mind it either way, as my impression is these special console types are not that heavily used by the great majority of users; I mostly suggested flattening it by one level since it seemed like there might be concern with that. If users do raise an issue about it, we can always make the change then.

I disagree. First, because it'd be a regression in functionality that is quite simple for us to support. And second, because you can have a console filled with results that you don't want to clear, but you could need to run another computation in the same kernel that can have lots of output. In that case, the most natural thing is to connect another client to the same kernel.

Well, it's not really a regression if its a minor UI hint indicator for a feature that, as far as I'm aware, few or no users actually use—despite helping probably thousands of Spyder users over the years I don't think I've ever seen any user, in any context, with multiple tabs connected to one console, whereas I have had more than one be confused what the "A" meant. Neither I nor @mrclary had ever used this despite being longtime heavy users of lots of consoles/kernels for half a decade, and I didn't even know what it meant despite writing our current detailed tutorial on connecting to local and remote kernels as well as answering user questions about it.

Regarding the use case presented, as someone who heavily relies on many thousands of lines of output history and long scrollback, I wouldn't remotely consider doing this; I'd just increase the number of lines of scrollback in the IPython Console preferences. It is far more "natural" to do that then a user manually figure out what kernel it is and then manually go through the steps to connect to it, with zero indication from the UI, docs or anything else that this is even possible let alone advisable.

That being said, as long as we only show B, C on the second, third, etc. consoles for the same kernel so it doesn't confuse the >99.9% of users who don't use this, I don't see much harm in it.

Are there other means to communicate these in addition to tab text? For example, icons, colors, font weight, or whatever.

As suggested above, I suggest we use the Sympy, Cython and Pylab icons to indicate these special types of consoles, and a tooltip to show the other information beyond distribution, environment or file name, Python version and console number.

If hover-text is possible, then verbose information displayed there may relieve some of the need to communicate more in the tab text.

I would be very surprised if its not, as the editor appears to use the same tabs and has tooltips on them, and its what Isabela suggested above.

Do we need to distinguish between consoles with identical identifiers? The set of information above will distinguish between consoles if any of the identifiers differ. But what if they are all the same: same console type, exactly the same environment, connected to the same kernel. Do we need to distinguish consoles under this circumstance?

The distribution/environment name and Python version should render them fairly unambiguous, but I still propose a sequential number when those two both match, for consistency with the current UI.

Which information is most important, which should be tab text, which should be an icon, which should be displayed elsewhere?

As mentioned above, tab text: file name, distribution or environment name, and Python version; type as the respective icon and everything else in a tooltip.

And I think using envs is still an advanced feature that perhaps not many people rely on.

Well, they really shouldn't be, because as we all know that is how users routinely break things.

When I think of my maths undergraduate students, the vast majority (>90%) have never installed a package because Anaconda comes with all that they need. They don't know about environments, they have no need to know about it, and I want us to keep beginners like this in mind. For these people, "Console", "Console 2" etc have proved to be working fine.

Okay, so in that case, we could just go with the Python X.Y[.Z] name for all consoles that aren't started in an explicit user-named including base and the Spyder internal environment.

This would work even better for beginners, I think, because it's a place for executing Python. However, it seems to contradict what you are proposing later on, when you say:

Anaconda base: Anaconda (Py3.7), Anaconda (Py3.7) 2, Anaconda (Py3.7) 3

I can easily imagine beginners confusing an IPython console named "Anaconda" in the tab with the "Anaconda prompt" on Windows.

The tension here is that for users that open consoles in different environments, we want to distinguish between the environments and thus mention the environment name. For users that don't know about environments, mentioning the environment name without context is confusing. Maybe there is no good solution, but maybe we can distinguish between these two classes of users. Here are some suggestions:

At start up, we look whether there are multiple environments. If not, we go in "beginner mode" and use a plain "Console" or "Python" in the tab name. If yes, we go in "advanced mode" and mention the environment name, like Ryan and CAM propose.
Alternatively, we don't mention the environment name for consoles opened in the default environment and we do mention the environment name for consoles opened in non-default environments. This may be what CAM is proposing two comments above here, but I'm not sure. The disadvantage here is that the text in the tabs are not consistent.
Alternatively, we don't mention the environment name if all consoles are opened in the default environment and we do mention the environment name as soon as any console if opened in a non-default environment. This means that if all consoles are in the default environment and then the user opens a console in a non-default environment, the text in the tabs of the existing consoles has to change. This may be confusing and also technically complicated.

However, it seems to contradict what you are proposing later on, when you say:

Right, those are two different proposals; the latter was more specifically tailored to address the types of situations @ccordoba12 posed in the quote I was responding to in that one, with a user that has, e.g. Python installed with both Homebrew and Anaconda, which is something users commonly report/open support requests for when they get confused which Python they're running in the IPython Console and why their packages installed in one or the other aren't showing up.

I can easily imagine beginners confusing an IPython console named "Anaconda" in the tab with the "Anaconda prompt" on Windows.

Hmm, that's a good point. We definitely do see users confusing the IPython Console and the Anaconda prompt (and more generally, the Python interpreter vs. their system shell) a fair bit, usually trying to run system commands (typically package installation) in a Python interpreter.

Considering this further, I'm now thinking that Python X.Y for non-env Pythons (and <env-name> (PyX.Y) or <env-name> (X.Y) for environments is probably indeed the best approach overall for the tab titles. It It makes the tab titles shorter, simpler, and more consistent between known and unknown distro names (though less so between env and non-env Pythons, unless for those we only show the Python version in the tooltip), as we can show the distribution name/type in the tooltip I.e.

{Python X.Y|<env-name> (PyX.Y)} [N if >= 2]

The examples boil down to basically 2 cases:

Anaconda base, Homebrew, system Python, Spyder default: Python 3.7, Python 3.7 2, Python 3.7 3 Conda environment: pandas-env (Py3.10), pandas-env (Py3.10) 2

Alternatively, we move the Python version for specific environments to the tooltip (and status bar?) as well, giving us:

{Python X.Y|<env-name>} [N if >= 2]

Anaconda base, Homebrew, system Python, Spyder default: Python 3.7, Python 3.7 2,Python 3.7 3 Conda environment:pandas-env ,pandas-env 2`

The one downside with either of these is Pythons from different installations that are the same version don't have unique names/get numbered in sequence by default, but that's still a substantial improvement over the entirely generic, redundant and less informative "Console" name.

An even better idea for more experienced users would be user-customizable console names. Users could type in their own a template with placeholders for the things @mrclary lists and that we would display in the tooltip (env name, env type, Python version, interpreter path, Console type, etc). But that's definitely more of a nice to have than a requirement.

For users that don't know about environments, mentioning the environment name without context is confusing. Maybe there is no good solution, but maybe we can distinguish between these two classes of users.

IMO, there is a fairly straightforward solution, as proposed above—for users who have named and created environments and told Spyder to use them (which would all have to happen for them to show up in the tab in the first place), show the env name (which the user gave the environment) in the console tab. Otherwise, show something more generic, like Python X.Y. That way, users who use environments get the env name, and those who are using the base environment, the "default" Spyder Python environment or another installed Python, do not.

At start up, we look whether there are multiple environments. If not, we go in "beginner mode" and use a plain "Console" or "Python" in the tab name. If yes, we go in "advanced mode" and mention the environment name, like Ryan and CAM propose.

Alternatively, we don't mention the environment name if all consoles are opened in the default environment and we do mention the environment name as soon as any console if opened in a non-default environment. This means that if all consoles are in the default environment and then the user opens a console in a non-default environment, the text in the tabs of the existing consoles has to change. This may be confusing and also technically complicated.

IMO, these both seem like a lot of complexity for relatively little gain over the more straightforward approach in my current proposals, and I'm unclear on the real gain vs. my most recent proposals of showing the Python version and/or distribution name for consoles not started in an environment.

Alternatively, we don't mention the environment name for consoles opened in the default environment and we do mention the environment name for consoles opened in non-default environments. This may be what CAM is proposing two comments above here, but I'm not sure. The disadvantage here is that the text in the tabs are not consistent.

Yeah, this is what I'm proposing. It's not perfect, but it wouldn't be that inconsistent using the Python version as the default "environment name" and the disadvantage seems very minor to me compared to the others.

An even better idea for more experienced users would be user-customizable console names. Users could type in their own a template with placeholders for the things @mrclary lists and that we would display in the tooltip (env name, env type, Python version, interpreter path, Console type, etc). But that's definitely more of a nice to have than a requirement.

I was about to recommend the same thing. In Preferences -> IPython console -> {Display|Advanced settings} we could have a format template that the user could customize. The default template could be as @CAM-Gerlach has suggested: {<env name> | Python X.Y} [N if >=2]. But I suppose this could be implemented sometime later.

So this is how I see the converging consensus so far:

Console tab tooltip will have verbose information, as listed above
Console status bar already shows <env name> (Python X.Y.Z), but should reflect active console tab and use an icon indicating type: IPython, SymPy, Pylab, or Cython
Console text: {<env name> | Python X.Y} [N if >=2]
Possible future enhancement: customized console tab text template

Thanks @mrclary for summarizing the discussion so far. My comments about it:

Console tab tooltip will have verbose information, as listed above

Ok, I think this is a good idea to quickly check any console env without clicking on it and then looking at the status bar.

Console status bar already shows (Python X.Y.Z), but should reflect active console tab and use an icon indicating type: IPython, SymPy, Pylab, or Cython

Updating the status bar widget with the console environment is a great idea! Thanks for suggesting that @mrclary. This would make the UX similar to the way the working directory toolbar works, in the sense that if you want to check what cwd (env) the console is in, you need to take a look at the toolbar (status bar).

However, I disagree with using icons on it because I'm sure almost all consoles would be IPython ones, so the information conveyed by the icon would be irrelevant most of the time. Furthermore, when switching consoles connected to different envs, we'd need to update the env used by the editor for completions so that users get the completions they'd expect per env. That way, the current env reflected in the status bar would the one used by the console and editor (and also the profiler/code analysis, etc), so using icons tied to the console wouldn't make sense (another advantage of that approach is that we'd be able to remove the Completions status bar widget)

Finally, I'd like to maintain that Pylab/Sympy/Cython consoles are simply named Pylab 1, etc. That way users would still know that they are special consoles (i.e. different from regular ones).

Console text: { | Python X.Y} [N if >=2]

Given that this information would be reflected in the status bar, I see little need to repeat it in the console tab name. So, I propose to leave that feature to be an option in the IPython console preferences, disabled by default. That would also have the advantage of keeping the interface really simple, which would go very well with what @jitseniesen said above:

For these people [i.e. beginners], "Console", "Console 2" etc have proved to be working fine.

And @CAM-Gerlach, please refrain from posting another long comment about what I just said if it's going to repeat your points above (which for me drags this kind of discussions to almost a dead end). I understand that you're very passionate about showing env names in console tabs, but I don't think that's the direction we're going to go by default. Instead, I prefer the simpler approach of updating the status bar widget to reflect the current env, name all consoles as Console 1, 2, ... and leave naming console tabs as envs to be an extra option, along with the template idea (which I think is a very good one for power users).

If users strongly request that they'd like to see consoles be named as envs, we can reconsider that approach (we know that when our users don't like something, they are very vocal about it).

However, I disagree with using icons on it because I'm sure almost all consoles would be IPython ones, so the information conveyed by the icon would be irrelevant most of the time.

I guess I don't really understand the objection to icons. Currently, the console status bar widget does not have an icon, while every other status bar widget does; so it should have an icon to be consistent with the paradigm. Even though most of the time it would be an IPython icon, this information is less irrelevant than "Console" in the console tab. Furthermore, when other console types are used, it conveys the same information in less space than "Cython" or "Pylab" in the console tab.

Furthermore, when switching consoles connected to different envs, we'd need to update the env used by the editor for completions so that users get the completions they'd expect per env. That way, the current env reflected in the status bar would the one used by the console and editor (and also the profiler/code analysis, etc)

That is correct, although I don't know if mean to argue in favor of this or against it.

so using icons tied to the console wouldn't make sense

I don't know how this follows from having to update the completions environment.

another advantage of that approach is that we'd be able to remove the Completions status bar widget.

I did not mention it in my previous comment, but I had the same idea. We could just eliminate the completions status bar widget altogether.

while every other status bar widget does; so it should have an icon to be consistent with the paradigm.

That's actually not correct. Some widgets have icons and others don't:

Furthermore, when other console types are used, it conveys the same information in less space than "Cython" or "Pylab" in the console tab.

This assumes that people have seen the Matplotlib or Cython icons before and can recognize them (I'm not even sure if I'd recognize the Cython icon before googling for it!). And the Sympy icon has a lot of detail, so I doubt it could be displayed correctly in such a small size. So, in this case I'd prefer to be explicit about the special nature of those consoles by "flagging" that in their names.

That is correct, although I don't know if mean to argue in favor of this or against it.

That's what I'd naturally expect: if I'm working in a console that has Scikit-learn, I'd like to see code completions for it, regardless of what my completions env is. Otherwise, I'd have to always recall that Spyder has a completions env that is different from the console env and which libraries it has.

This assumes that people have seen the Matplotlib or Cython icons before and can recognize them (I'm not even sure if I'd recognize the Cython icon before googling for it!). And the Sympy icon has a lot of detail, so I doubt it could be displayed correctly in such a small size. So, in this case I'd prefer to be explicit about the special nature of those consoles by "flagging" that in their names.

Ok, counter proposal to my own proposal so that we can name consoles only as Console X: what if instead of the project icons, we use C (i.e. mdi.alpha-c-box or similar) for Cython, S for Sympy and P for Pylab? Also, we should mention the console type in the tooltip's widget.

Saw this trying to use Spyder, want to put in my own experience.

When using Thonny, I can go to the menu, pop out a terminal in the current venv, use a GUI in Thonny to search for and install packages, create a new venv, add an existing venv (so it runs the activate script), etc.. There's a dropdown in the bottom-right to pick an interpreter (it even detects if micropython is installed on a connected device like a Pi Pico).

When I opened Spyder, I immediately looked for how to add a virtual environment. It lets me pick a path to an interpreter in the options dialogue, but it isn't clear to me that this will activate the venv (I'm assuming it won't). I looked for a way to get a terminal to add packages with pip, but it wasn't obvious how to do that in the 2 minutes I spent trying. I also couldn't locate the built-in package manager, if any.

I checked on Google and found a lot of stuff that said to install Spyder into each Anaconda environment; I'm running Spyder installed from its installer, not conda. I found spyder-env-manager, which said to install it using pip, which again I don't know how to install packages in Spyder's environment (and don't know if that would actually add the plug-in to Spyder).

Spyder looks nice, but I went right back to Thonny because I can't figure out how to install numpy. I loaded my Pi Pico with a custom-compiled version of Micropython with ulab; doing that was more obvious than figuring out how to run pip in Spyder's Python environment.

Not being able to install or select a venv (putting aside being stuck on Python 3.8, which is its own problem) and manage packages (by getting a terminal open in the venv, or by having a GUI to manage packages) is the only thing blocking me from using Spyder right now.

From my perspective, the lack of a venv manager and some obvious way to install packages (even getting a terminal open is either non-obvious or not in Spyder—there is a plugin for this, the only thing online about installing Spyder plugins has to do with already being in a terminal for the Python environment Spider's running in, this is a causality loop that may very well destroy the entire universe) is the biggest barrier to accessibility. Spyder becomes "you can use this IDE only with the special environment it comes with and only with the default Python modules" even for moderately experienced developers, which is completely useless (hence being a barrier to accessibility). As far as I can tell, Spyder only really works for conda users.

spyder-ide / ux-improvements

Add menu to change console environments #10