Global shim's dependency management is confusing and non-intuitive

[MatthewScholefield]

There are some important open questions around using rye to manage the global Python version. Notably:

• https://github.com/astral-sh/rye/discussions/998
• https://github.com/astral-sh/rye/discussions/489
• https://github.com/astral-sh/rye/discussions/1066
• https://github.com/astral-sh/rye/issues/651#issuecomment-1946323571

Given these open questions and the confusion it creates, I wonder if the global shim option should be removed, at the very least from the automated installer. What do you think?

Alternatively, if we get some docs on the global shim and know how to install packages and run ipython then that would also solve this issue.

[bluss]

Not a replacement for docs, but there is a video on how the rye shims work https://www.youtube.com/watch?v=TKnYcyz095g by Armin

Existing docs are under https://rye-up.com/guide/shims/ - it would be great to be detailed about what's missing.

One thing I can spot now is that the docs make it sound as if the python +VERSION feature requires global-python to be active, but it does not :slightly_smiling_face: (which is a nice feature IMO)

[MatthewScholefield]

Ultimately, I think I realize now that rye's method of managing the global Python version isn't actually a 1:1 replacement for the normal system of managing a global Python installation. From what I understand (I may be wrong), rather than maintaining a single, global Python environment which is what most people would expect, it creates a new isolated environment for every new package.

For simple self-contained Python scripts that you want to install globally, this is amazing. However, a common reason many people use the global Python environment is for the interactive REPL and for running quick scripts that have minimal dependencies. For these purposes, the global Rye shim is not only unsuitable, but actively confusing. Here's why:

1. python3 is quite useless since you can't actually add global library dependencies and run simple global scripts.
2. Similarly, if you do rye install ipython, it creates a new fresh environment with only ipython removing most of the value of ipython. You can continuously run rye uninstall ipython && rye install ipython --extra-requirement numpy but this is quite cumbersome. A similar problem also exists for jupyterlab.
3. Not knowing anything about this leads to a huge amount of confusion because you have all of the following ways of installing packages that all work in subtly different ways:
   • pip3 install ...
   • python3 -m pip install ...
   • rye install ...
   • ipython3 -m pip install ...

Now combine these on top of permutations of running rye install pip, rye install ipython, sudo apt-get install python3-pip, sudo apt-get install python3-... etc. and you can see it becomes a nightmare, especially for someone who is just getting started with Python.

Overall for these reasons, I think we really should avoid giving users an easy accidental pathway into this hell to maintain the project's mission of "A hassle-free experience for Python developers at every level." :joy:

If we wanted to make this easier, I think it's possible, but it just involves building a simple supported CUJ for these use cases (ie. rye install-dep python3 numpy) and preventing people from shooting themselves in the foot with it. Personally though, I would lean towards making the whole independent-env-per-package thing be some very specific command like install-executable and leaning on a shared global env by default simply because that's kind of what the Python ecosystem has arrived upon for better or for worse.

[bluss]

Note that rye install is an alias for rye tools install. It's the tools feature, like a lite version of what pipx does. Rye install is not for installing packages that you import anywhere else.

That's a different feature than global shims for python which I thought this was about - best to clarify the issue.

[MatthewScholefield]

The issue is just about the entire rye global shim experience leading to confusion. I think I might have conflated the two (good catch).

In general I think it's because it's unclear how to install dependencies on the rye-managed Python versions using global shims which would lead people to using rye install, further complicating the situation.

[bluss]

Good point, Rye needs to clarify that. I'm not sure what the design is, but it looks like global shims only offer python toolchains and no explicit (--system/--user type) package install. For those that only use virtualenvs, that's not a problem.

[frandmb]

Note that rye install is an alias for rye tools install. It's the tools feature, like a lite version of what pipx does. Rye install is not for installing packages that you import anywhere else.

That's a different feature than global shims for python which I thought this was about - best to clarify the issue.

The documentation is clear but the feature is (sometimes) confusing. I'd expect rye install/remove [package] or a pip install [package] shim to manage global, shared packages when global-python is set to true.

I agree with everything @MatthewScholefield said regarding it over-complicating situations when you want to use packages outside of a Rye project. Maybe you've got some tooling on a folder you don't want to write to, or want to run the REPL on a non-managed directory. Similar to what npm install -g does for Node modules.

[SichangHe]

The question is, when I need to run a one-off Python script, I don't want to rye init a project—I need a global venv—so how do I do that with Rye?