What would you like to see in proto?

milesj commented 8 months ago

Hey everyone,

At this point in time, proto is relatively stable, and we believe an official v1 is around the corner. Before we do that though, we want to hear from the community about what they would like to see in the project. We have a few ideas of our own (below), but we want to make sure we're not missing anything useful.

We want to gather this feedback now incase we need to introduce breaking changes, which we'd like to do before v1. We're also open to collaborating on these features, so if you're interested in working on something, let us know!

Current roadmap

Build from source

Building from source is required to support additional languages, like PHP and Ruby, but can technically be used for any language as an alternative to pre-builts. We've started working on this, but it's quite hard to nail down a Rust/WASM API since every language builds from source differently.

Unlocks:

New languages to support.
Can customize what features/extensions are enabled for the language (for example, PHP may want mysql).
Supports versions of languages that may not have pre-builts.

Blockers:

Not enough research on how every language builds from source.
How to distinguish versions with different requirements?
- For example, 1.2+mysql and 1.2+pgsql.
- Both for configuration and folder structure.

More built-in languages

We would like to support more languages out of the box, primarily web-based languages like PHP, Ruby, and Python (provide better support). Other languages outside of this list will be supported by the community.

Blockers:

Requires building from source.
Someone with better knowledge of these languages.

Backlog

Plugin registry

Right now, plugins are supported using file system paths or secure URLs, but both of these have limitations. What proto really needs is some kind of central registry in which the .wasm files can be stored and accessed. This will also allow for metadata, versioning, and self-management by users.

Unlocks:

Single source of truth.
Can search for plugins (proto plugin search).
Can associate metadata to plugins (name, description, author, version, etc).
Avoids having to configure [plugins] (for the most part).
Users can self-manage their plugins (publish, update, delete).
Maybe have a nice browser UI.

Blockers:

Have to build the registry from scratch (not aware of an off the shelf solution).
May be costly to run (storage, bandwidth, etc).
How does this work with TOML plugins?
Is it worth it?

Lockfiles/checksums

We would like to generate lockfiles and checksums for all downloaded plugins. This is difficult with our current implementation, as no checksum/metadata is associated with file system paths or secure URLs when downloading plugins.

Unlocks:

Ensure the safety of our users and their machines.
We avoid executing potentially malicious .wasm files (is this even possible with WASM's sandbox?).

Blockers:

Most likely requires a plugin registry to be effective (unless someone has a great idea/alternative).

Phault commented 7 months ago

We avoid executing potentially malicious .wasm files (is this even possible with WASM's sandbox?).

Unfortunately it definitely is:

exec_command allows arbitrary commands
Sensitive environment variables could be dumped and sent with a HTTP request
The download url for the prebuilt binaries could point to malicious payloads
I'm sure write access to env variables can also be abused somehow

So lockfiles/checksums are definitely a must as a minimum. On top of that we might want to look into something like mise's paranoid mode, though I don't think that really achieves much.

PointSingularity commented 7 months ago

I don't know if this something that is wanted, but having the option of forcing a tool would be nice.

By that I mean, if I have for example yarn pinned, and someone tries to do a npm i, that is allowed. If there was an option to have a setting that either redirects the command to yarn, or errors out when the right tool is not used would be great.

Firesz25 commented 7 months ago

I would like to use proto as a library. To be used as one of the options for installing packages.

milesj commented 7 months ago

@Firesz25 proto is a Rust library: https://crates.io/crates/proto_core Are you looking for something else?

prabirshrestha commented 6 months ago

I would like proto to support features from mise and fnm. I personally use fnm because it works on all OS (linux/mac/windows).

I like to have this on my shell profile.

if hash fnm 2>/dev/null; then eval "$(fnm env --use-on-cd)"; fi

Whenever I cd into different folder it auto detects .nvmrc file and selects the correct node version.

$ which node
/Users/prabirshrestha/Library/Caches/fnm_multishells/40464_1714723038926/bin/node

As you can see the path, we can have different tabs it for different projects and it will correctly use the right version of node. This is probably better than just having a global ~/.proto/bin. README.md on mise talks a bit about it this where it avoids having shims and uses the real path of node.

ImBIOS commented 6 months ago

little elephant, PHP :v . . . I'm not joking ✌️

milesj commented 6 months ago

I've thought about PHP a lot, but it's very complicated since it's all build from source.

RotaN8 commented 6 months ago

I have a couple of ideas:

A command to update all the tools that have been pinned as latest (globally). I have actually done a workaround by soft linking the .prototools file from ~/.proto/.prototools to ~/.prototools to be able to do proto use and update all the tools that I manage with Proto. It's a great tool manager 😄. I know this might not be the initial purpose of Proto, but I've found it useful when used this way.
An option in the config file to limit the versions that are kept. For example, I want to have only the latest version of each tool I use. That feature would be useful, and I think it's not difficult to implement.

Thanks for this amazing tool ❤️

milesj commented 6 months ago

Both of those seam reasonable, will look into!

MRZ07 commented 5 months ago

Java SDK Support

nekoleamo commented 5 months ago

in now version, must install node and npm separately, can it install the most suitable version of npm automactically when installing node

milesj commented 5 months ago

@nekoleamo You need to enable this setting: https://github.com/moonrepo/node-plugin?tab=readme-ov-file#nodejs

All settings were changed to false by default, as most users want things to be opt-in.

prabirshrestha commented 4 months ago

Just tried the new eval "$(proto activate bash)" and I have replaced mise. Will see how it goes over in the next few days.

Here are few more suggestions.

Would be good if proto has support for mise ls equivalent (Plugin/Version/Config Source/Requested version). Or should proto status supposed to show the .nvmrc? Currently it only shows moon as it is defined in .prototools.

mise commands seems more ergonomic.
- For example it auto aliases mise ls to mise list.
- mise install auto installs all tools. proto install would be convinent over proto use.

$ mise install node@20.0.0  # install specific node version
$ mise install node@20      # install fuzzy node version
$ mise install node         # install version specified in .tool-versions or .mise.toml
$ mise install              # installs everything specified in .tool-versions or .mise.toml

Should the appropriate version of npm be installed by default when using node? I have some projects that I use rush which means I need to use proto install npm && rpm install -g @microsoft/rush when setting up a new machine.
Add proto to homebrew instead of manually searching how to install proto.

milesj commented 4 months ago

Would be good if proto has support for mise ls equivalent (Plugin/Version/Config Source/Requested version). Or should proto status supposed to show the .nvmrc? Currently it only shows moon as it is defined in .prototools.

Only .prototools right now. Getting the other files right now is a bit tricky.

For example it auto aliases mise ls to mise list.

Almost all proto commands have shorthand aliases also.

mise install auto installs all tools. proto install would be convinent over proto use.

This is a nice idea, easy enough.

Should the appropriate version of npm be installed by default when using node?

This has to be enabled manually. https://github.com/moonrepo/node-plugin?tab=readme-ov-file#nodejs

Add proto to homebrew instead of manually searching how to install proto.

Supporting homebrew is quite difficult and annoying, and honestly, I don't have the time or energy for it. However, there is an unofficial one: https://formulae.brew.sh/formula/proto#default

W1M0R commented 4 months ago

I've only very recently discovered proto, and I really like the experience so far, especially its simplicity of use, good documentation, and easy way to contribute packages.

I'd like proto to remain simple and I like the approach taken with the rust plugin, where it doesn't re-invent the wheel and piggy backs on rustup.

That being said, I'd like the python story on proto to be better, but I wouldn't want proto to get bogged down by all the complications of that ecosystem. So my request would be that the python plugin be improved, by using existing python-centric systems, more specifically, via pixi. Pixi is also a rust based project and uses Conda Packages. I decided against pixi for general purpose utilities, but for Python packages, it really seems unmatched.

W1M0R commented 4 months ago

It would also be great if there can be a detection strategy that just reverts to the global install of a tool after all other avenues have been tried.

From what I can see, if proto doesn't find a tool in its own config, e.g just, and you try to use just, it will fail with an error, even though I have just installed globally. I could just re-order my PATH to ensure my global just is considered first, but this would cause other problems. I could also add just to the root prototools file and enable Auto-Install. But it would be nice to opt in to a fallback strategy that allows the command even though proto couldn't find it using its own detection method.

I probably experience this because one of the subprojects in my monorepo installed just via its prototools file, whereas other projects in my monorepo don't use proto yet, so they expect the globally installed just, but since the proto shim is already in my PATH, it gets the first chance to execute just and then rejects the execution.

In retrospect, once the native Windows path lookup mechanism has resolved a certain tool to a proto shim, there is probably not much that proto can do. One method would be to re-execute the command, but from a modified PATH environment that excludes the proto shims and bins, but I foresee other issues with this method.

I'm just trying to figure out how to prevent proto shims from affecting projects that don't use proto. This will probably involve installing proto with a setting that will prevent it from updating the global user PATH environment (in a similar way where None can be specified during installation to prevent updating of the PowerShell profile). And then letting projects that want to use proto shims, manually add the proto shims temporarily to a "dev shell" environment.

I'm also probably doing something wrong.

milesj commented 4 months ago

@W1M0R

That being said, I'd like the python story on proto to be better, but I wouldn't want proto to get bogged down by all the complications of that ecosystem. So my request would be that the python plugin be improved, by using existing python-centric systems, more specifically, via pixi. Pixi is also a rust based project and uses Conda Packages. I decided against pixi for general purpose utilities, but for Python packages, it really seems unmatched.

I took a look at pixi and this seems like it's more package focused? Where as the proto python plugin is focused entirely on just installing python itself. With that said, I'm looking to improve python in proto 0.39, probably by using python-build under the hood (althought this doesn't support Windows).

I don't use python, so trying to do it all myself has been an uphill struggle.

It would also be great if there can be a detection strategy that just reverts to the global install of a tool after all other avenues have been tried.

It does work this way. Are you sure there is a version entry in ~/.proto/.prototools?

I'm just trying to figure out how to prevent proto shims from affecting projects that don't use proto. This will probably involve installing proto with a setting that will prevent it from updating the global user PATH environment (in a similar way where None can be specified during installation to prevent updating of the PowerShell profile). And then letting projects that want to use proto shims, manually add the proto shims temporarily to a "dev shell" environment.

I've also been thinking about this. There's a proto deactivate on the roadmap, but it may be a while to flesh out correctly.

W1M0R commented 4 months ago

@milesj

I took a look at pixi and this seems like it's more package focused? Where as the proto python plugin is focused entirely on just installing python itself. With that said, I'm looking to improve python in proto 0.39, probably by using python-build under the hood (althought this doesn't support Windows).

It is more package focused, but it also installs a Python interpreter, without any platform restrictions (i.e. it works on Windows): https://pixi.sh/latest/tutorials/python/#whats-in-the-environment

Let me start off by saying that Python is not my primary programming language, although I have done a few small things in Python before. I do like using Python cli tools. One example is mkdocs and mkdocs-material. With Pixi, I can install those Python cli tools in my environment, and know that it would work as intended in Windows and Linux. I'd imagine that with proto one would also be able to add mkdocs, mkdocs-material and any of the wide variety of mkdocs plugins with their runtime dependencies. Pixi manages that complexity (because the Conda ecosystem does). Conda sounds like a de-facto standard in the Python community, so if Proto wants to reach that audience, it might be helpful to have some kind of Proto-level integration with Conda/pixi.

However, this might be out of scope for Proto, and probably not necessary for a 1.0 release. Perhaps it is good enough to add the pixi cli tool as a proto toml plugin, and then users can add pixi to their prototools and use that for the python specific stuff.

dudicoco commented 3 months ago

I would like to see a better story for Proto - what makes it unique?

There are other tools which do more or less the same thing - aqua and vfox are the ones i'm aware of.

These tools, like Proto, have definitions on how to download a binary and extract it, and provide registries with tools definitions:

There are also tools which are wrappers for nix, for example devbox - rather than having to build and maintain a registry of tools/plugins you get out of the box access to all of the nix packages.

meop commented 1 month ago

Thank you for this project. I like how clean it is, supporting shims, binaries, shell activation, all well documented and each behavior optional.

I tried several competitors in a row, but each had a few behaviors I did not like.

One behavior that I do make use of from Homebrew, is that its bins/symlink approach can have multiple versions of a binary in linked at the same time. For example:

/usr/local/bin/python3.9 -> ../Cellar/python@3.9/3.9.20/bin/python3.9 /usr/local/bin/python3.10 -> ../Cellar/python@3.10/3.10.15/bin/python3.10

This is similar to how I install multiple versions of Python at once on Windows.. both sets of binaries reside on the PATH at once, but have different local symlink names, pointing to the real, not-version-named binaries.

In a project where it is useful to use or test different versions of a binary, you can currently just do 'proto run ', but to make that a little simpler, proto could instead create links named like 'bun1.1.29' and 'bun1.1.30' both residing in the ~/.proto/bin directory, in addition to a singular, unversioned 'bun' link/binary.

I have not thought out this feature thoroughly, and it may even be an anti-pattern, but it is an approach I have been using. :)

milesj commented 1 month ago

@meop This is actually a great idea. I've been wondering what to do with ~/.proto/bin since it's kind of an after thought, and this seems like a perfect solution.

adaliszk commented 1 month ago

I would join to dudicoco's suggestion, having an integration with Nix packages would be awesome and also a major enabler.

milesj commented 3 weeks ago

@meop This landed in v0.42.

moonrepo / proto