dependencies

[jdx]

Plugins need the ability to declare dependencies on other plugins. Exactly how this would be defined is unclear, and it will be fairly complex to try to implement it, but I think we will eventually need it.

This is a problem currently for plugins like rtx-poetry which will call python3 on PATH. Presumably some people may want that python to be installed via rtx. Right now because these get installed in parallel, it would almost certainly fail the first time since python takes a lot longer to install.

For now, you'll need to run rtx i python if you have this problem to explicitly install python first.

However long term, poetry should be able to declare that python is a dependency. This would likely go into rtx.plugin.toml, I can think of a few different ways to represent this:

# this is naive, it wouldn't allow different versions of poetry to require different versions of python
[dependencies]
python = {version="^3.11,^3.12"} # semver selector (not the same as typical versions)

# this uses semver to set dependencies for a particular version
[dependencies.^3]
python = {version="3.11", optional=true} # optional means it will use the system python unless python is used in `.rtx.toml`

Alternatively a script might be a better way to represent this:

#!/usr/bin/env bash

if [ "$RTX_POETRY_VERSION" = "1."* ]; then
  # poetry is version: 1.*
  echo "python ^3.10,^3.11"
else
  echo "python ^3.12"
fi

I'm thinking rather than putting these things on PATH we'd just make env vars like RTX_PYTHON_PATH which can be used to find their location. This way we keep PATH pristine since a plugin's dependencies might not be the same as what is in .rtx.toml.

[tomassatka]

hi @jdxcode I came across the same issue when i define plugin JIB that require to have already installed java. Or would like to create custom plugin for octo cli that already need to have dotnet be installed. Basically facing the same problem and that i the declaration of dependencies inside singe .tool-versions

In bigger picture i am trying to build docker images based on rtx installation.. here is a small snippet that shows the gist:

RUN chmod -R a+rx /root &&\
    bash /.asdf/build-deps/build-deps &&\
    if ls /.asdf/build-deps/**/build-deps 1> /dev/null 2>&1; then for script in /.asdf/build-deps/**/build-deps; do bash $script; done fi &&\
    rtx install &&\
    for bin in $(rtx bin-paths); do cp -s -r $bin/* /usr/local/sbin; done &&\
    bash /.asdf/post-build

I had also an idea how to go around of defining the dependencies and that is to declare a state in .tool-versions a flag that would tell that installation is optional and if failed that it would not return exit code -1. Then it would require to have some way of listing if all is installed maybe rtx current to iterate few times... lets say give it a max of 5 iterations and re-exeucte rtx install until all installed.

Yea or nay? tom

[jbadeau]

I see the need for a real dependency RFC but this should be aligned with the asdf guys prevent fragmentation and drive the best solution.

In the short term why not a simple solution like

kubectl 1.26.1
helm 3.11.1
helmfile 0.151.0
# rtx-layer-01
nodejs 18.14.0
yarn 1.22.19
python 3.11.2
dotnet 6.0.407
# rtx-layer-02
sops 3.7.2
yq 4.28.2
java zulu-19.32.13
glab 1.26.0
# rtx-layer-03
jib 0.12.0

RTX would interpret a well known comments like rtx-layer and install layer by layer. This would not break asdf and would provide a quick and easy way to solve the issue from the RTX side.

[jdx]

I had also an idea how to go around of defining the dependencies and that is to declare a state in .tool-versions a flag that would tell that installation is optional and if failed that it would not return exit code -1. Then it would require to have some way of listing if all is installed maybe rtx current to iterate few times... lets say give it a max of 5 iterations and re-exeucte rtx install until all installed.

This is optimizing for the wrong part I think. It's trivial to add a dependency field into rtx.plugin.toml to represent them, what's hard is the resolution and installing in parallel. What you described though is roughly how I think it needs to work. (I'm probably not going to even consider circular dependencies for a long time if ever).

RTX would interpret a well known comments like rtx-layer and install layer by layer.

I'm against this for a few reasons. First, it requires the end user (not plugin author) to define. Second, it is temporary and would eventually need to be removed. Even as an experimental feature that could be painful.

I appreciate the suggestions, and welcome more, but I think we need a proper solution here. One thing that would help is if you knew what to even call this algorithm that I need. I basically will have a tree data structure and need to pop the leaves (and only the leaves) off until everything is gone. Every package manager needs to do this, so I'm sure there is prior art out there but I'm not sure what to even search for with Google. It doesn't need to be efficient since it's not like people will ever have that many tools installed with rtx (at least not in any near future).

[jdx]

One thing I was thinking is this might be sort of possible today—though it's not a use-case I've designed for or have tested myself. But what if you just did rtx i node@someversion inside of a plugin's bin/install? Perhaps you could also use the shebang thing in the README as a shim to call your CLI with rtx x nodejs@someversion.

You could also get the binary location with RTX_NODEJS_VERSION=someversion rtx which node. I should probably also make it possible to do this with rtx which nodejs@someversion node, but that doesn't seem strictly required here.

[jdx]

I think this might be what I'm looking for: https://docs.rs/petgraph/latest/petgraph/graph/struct.Graph.html#method.externals

That gives me an iterator of a graph of all of the nodes that don't point to anything (i.e.: don't depend on anything). I can wrap this in a mutex and pop off each one after installing.

[jdx]

at least for poetry (the most commonly asked for), I think I've found a simple solution that won't require a complex dependency feature. What I did was just load whatever other bin_paths are available in the exec_env script. This effectively exposes a python managed by rtx to the poetry plugin. It may mean that you need to install python explicitly before installing poetry though.