Packages not found (cannot be excecuted)

[RoyRao2333]

What version?

0.39.2

Which command?

No response

What happened?

Sorry for the fuzzy title... It's quicker to take a look at these screenshots:

---

For Node:

But I can find npm and corepack in ~/.proto/tools/node/20.16.0/lib/node_modules:

---

For Python, packages installed by pip cannot be found either:

❯ which python
/Users/royrao/.proto/shims/python

❯ python --version
Python 3.12.4

❯ pip list cyberdrop-dl
Package              Version
-------------------- -----------
cyberdrop-dl         5.3.31

❯ cyberdrop-dl
zsh: command not found: cyberdrop-dl

I can only use it if I export ~/.proto/tools/python/3.12.4/install/bin to PATH.

---

This is too much trouble. How can I fix this? Thanks!

Trace logs?

No response

Operating system?

MacOS

Architecture?

arm64

[milesj]

@RoyRao2333 A few things:

• Node doesn't install npm. Just because npm is shipped with Node doesn't make it available in proto. You need to also install npm with proto.
• proto shims don't apply to every binary that a tool has, just the primary one (and sometimes some secondary ones). This is especially true for packages. It's not possible to create a shim for every package binary.

With that said, if you want all of this to "just work", you need to use activation. https://moonrepo.dev/docs/proto/workflows Activation will only enable tools that have a version defined in a .prototools file.

[RoyRao2333]

@milesj

Thanks for the reply.

And,

1. Can I safely delete npm and corepack folders in lib? (I'm a bit of a neat freak, sorry)
2. I read the doc and added the command to the end of my .zshrc:

# proto
export PROTO_HOME="$HOME/.proto";
export PATH="$PROTO_HOME/shims:$PROTO_HOME/bin:$PATH";
eval "$(proto activate zsh)"
# proto end

But still, I can't invoke cyberdrop-dl (already sourced .zshrc). Is there anything missing or am I getting this wrong? Please give some more specific guidance on how to use packages installed by pip?

Thank you very much for your time!

[RoyRao2333]

Activation will only enable tools that have a version defined in a .prototools file.

I do have a ~/.proto/.prototools file:

But as far as I know, these versions are from the packages managed by proto, not by pip. Should I add package versions installed by pip? If do, how can I properly do that?

[milesj]

Activation will include pip packages on PATH because they belong to the python tool.

[RoyRao2333]

@milesj

Thanks.

So as you can see, I've already appended eval "$(proto activate zsh)" to .zshrc. Why still not working?

I'm not quite understand the principle of Shell Activation, sorry for the inconvenience.

[milesj]

Activation updates your PATH so all of these binaries are available. Globals (~/.proto/.prototools) are not enabled by default so you need to pass --include-global.

[RoyRao2333]

@milesj

# .zshrc

# proto
export PROTO_HOME="$HOME/.proto";
export PATH="$PROTO_HOME/shims:$PROTO_HOME/bin:$PATH";
eval "$(proto activate zsh --include-global)"
# proto end

I even reinstalled cyberdrop-dl and still no luck...

[milesj]

Are you reloading your shell after making changes to the config?

What's the output of proto activate if you run it manually?

[RoyRao2333]

@milesj

Yes, I reload shell everytime.

Here's the output:

❯ proto activate zsh --include-global
export __ORIG_PATH="$PATH"

_proto_hook() {
  trap '' SIGINT
  output=$(proto activate zsh --export)
  if [ -n "$output" ]; then
    eval "$output";
  fi
  trap - SIGINT
}
typeset -ag chpwd_functions
if (( ! ${chpwd_functions[(I)_proto_hook]} )); then
  chpwd_functions=(_proto_hook $chpwd_functions)
fi

[milesj]

Sorry, output of proto activate --export. It will show you the PATH being set.

You can also set PROTO_LOG=trace before activate to see what it's doing.

[RoyRao2333]

@milesj

Here's the full output:

PROTO_LOG=trace.txt

[milesj]

@RoyRao2333 I see /Users/royrao/.proto/tools/python/3.12.4/install/bin in your PATH because of activate. Be sure that you are installing pip packages using the proto version of pip. which pip should point to the python 3.12.4 version.

Edit: Try cding somewhere or calling _proto_hook directly. I think there may be a bug where it doesn't evaluate till after you move around.

[RoyRao2333]

@milesj

Yes, I'm sure that I'm using pip managed by proto. Here's the full output:

As you can see, .proto/tools/python/3.12.4/install/bin is not added to my PATH variable.

[milesj]

@RoyRao2333 Yeah something seems off. Let me dig through the code and see if I can find the bug.

[milesj]

I think this will fix it. It's hard to verify some of this stuff. https://github.com/moonrepo/proto/pull/572

[milesj]

Can you try the latest patch?

If it still doesn't work, could you do the following:

1. Add PROTO_LOG=trace in your shell, before the activate command.
2. Add the eval + activate command snippet in your shell.
3. Open a new terminal (it should show some logs of activate triggering).
4. CD into a new directory (there should be more logs).
5. Verify PATH.

Then send over the logs :P

[RoyRao2333]

@milesj

3. Open a new terminal
4. CD into a new directory
5. Verify PATH

Thanks, it's working after these steps in v0.39.4 :)

Still got some questions here though:

1. Why must I change directories before this take effect? Wouldn't it be better to make it work once shell instance is initiated?
2. As you can see in the screenshot, .proto/shims and .proto/bin is duplicated because I added "$PROTO_HOME/shims:$PROTO_HOME/bin" to my PATH manually in my .zshrc before activate. But If I don't do that, proto can't be found at all (command not found: proto).

Thanks for you patience 😊

[milesj]

1. The hooks right now only fire on cd. The plan is to have it work when shell is activated, but it really depends on if there's a .prototools file and where a new terminal is opened in. There's also a few perf issues to work out. You could always call _proto_hook manually in your shell to trigger it for now.
2. Activate includes the paths just in case. We can probably filter them if they already exist.

[RoyRao2333]

@milesj

We can probably filter them if they already exist

I'd like this very much!

The plan is to have it work when shell is activated

I'll call _proto_hook for now. Hoping it'll work in the future releases.

Again, thanks for this excellent tool you guys brought us! Closing this issue now.

[milesj]

I was looking into removing the duplicate shims/bin paths, but it turns out it was added on purpose, and it's to ensure that proto shims take precedence over the injected tool bin directories.

If you don't care about that, you can pass --no-shim and --no-bin flags to the activate command.

[RoyRao2333]

@milesj

Got it. Thanks!

I'll try --no-shim and --no-bin now.