Cross-plugin dependencies?

[halostatue]

I’m looking at developing a variation of the kx tool (a fork of kiex) that I had been working on as an rtx plugin in order to provide a better replacement for the existing Elixir plug-in, since there are some limitations involved with how Elixir (and other BEAM-based languages) work in relation to Erlang. I’ll try to explain, since it’s a little wonky if you’re not in the ecosystem.

The current Elixir plug-in only allows downloading precompiled versions of Elixir, but those versions include ones that specify a major version of OTP (e.g., 1.15.7-otp-25 and 1.15.7-otp-26; if you specify 1.15.7 by itself, you get a version that is equivalent to 1.15.7-otp-25). This matters because if your Elixir is built for otp-25 (Erlang 25), you will not have features that may only be available when built on otp-26 (Erlang 26). It also matters because if you try to run either of these with otp-24 (Erlang 24)…it will not work.

So the plugins erlang and elixir have a dependency relationship that is not currently expressed (it is a limitation of the existing ASDF plug-ins). The erlang plugin is OK as it is based on kerl, but the Elixir plug-in is based in part on exenv, which has not been maintained in many years.

I’m looking for at least one of the following:

• When my new "elixir" plug-in is installed, I want to force the installation of the "erlang" plug-in. You cannot use Elixir without an Erlang environment, so it makes no sense to have one without the other.
• I want to allow more complex version strings like elixir@1.15.7-otp-26.1.2 (not a currently supported version by the prebuilt versions).
  • On detecting that 1.15.7-otp-26.1.2 is not a currently prebuilt version, I want to be able to ask the erlang plug-in whether 26.1.2 is installed.
  • If not, I want to ask the erlang plug-in to install 26.1.2.
  • After otp-26.1.2 is installed, I want to use that Erlang version in a build environment to build Elixir 1.15.7.
  • I want to be able to activate both otp-26.1.2 and elixir 1.15.7-otp-26.1.2 since I have built this version of Elixir with that version of Erlang/OTP.
  • (This would also allow the installation of special case versions like main@otp-26.1.2 as 1.16.0-dev1.)
• I want to be able to error out if attempting to use incompatible Elixir / Erlang versions.

This may not be possible right now, but I’m not sure how it would be done if it could be, or if this is something that you would be interested in offering extended capabilities in rtx for this sort of linked plugin capability.

(I could develop a plug-in that manages both things at the same time, but that gets a bit messy and would require reimplementing a fairly large part of kerl, which is unnecessary. It would also mean that if I have multiple Elixir versions that use the same version of OTP, I would end up duplicating the Erlang/OTP installation multiple times unnecessarily. I’d like to use the builds that are already provided by other systems.)

[jdx]

It's a long-standing gap of rtx, but yes, at some point this should be supported https://github.com/jdx/rtx/issues/393

[jdx]

1047 is at least a partial solution here. With that in place users can at least reuse an existing erlang install.

You cannot use Elixir without an Erlang environment, so it makes no sense to have one without the other.

perhaps, but I'm not sure rtx is the right tool to enforce this kind of relationship. I think when you want to strictly define each tool down to each dependency until you get to a common base there is a great tool for that already: nix.

I think rtx is useful for people that don't want that sort of pattern. For when you want your system dependencies used but then install things on top. I get that erlang+kx/elixir is a unique relationship though.

Another option might be building on something like I talk about in #1038

[halostatue]

Well, Elixir, Gleam, LFE, etc. all require an underlying Erlang baseband. It would be the same with Scala, Clojure, etc. requiring an underlying JVM baseband.

nix is really not a great tool for this stuff—not least of which because the nix language is only slightly better than the syntax of Dockerfile (I exaggerate — the nix language is excessively complex, but the Dockerfile syntax is excessively simplistic). At least the last time that I tried it, the experience on macOS was nearly unusable compared to the tools I was already using.

Going with the Elixir setup because I can describe it well, each version typically supports at least two major versions of Erlang (1.15 supports OTP 24, 25, and 26). I would love to have some way, from within an rtx plugin (I do not care about asdf compatibility), to basically say:

# assume that I have written a version parser — I have for other projects, even with bash
# this would take elixir@1.15.7-otp-26.1.2 and extract `otp-26.1.2`.
otp_version="$(parse_otp_version $provided_version)"
otp_major="$(parse_major_version $otp_version)"
# this would be 1.15.7-otp-26
elixir_version="$(parse_elixir_version $provided_version)"
elixir_otp_major="$(replace_otp_major $elixir_version $otp_major)"
elixir_version_only="$(remove_otp_major $provided_version)"

if rtx has erlang@$otp_version; then
  rtx use erlang@$otp_version
  install_elixir $elixir_version
elif has erlang@$otp_major; then
  rtx use erlang@$otp_major
  install_elixir $elixir_otp_major
elif has erlang@26; then
  install_elixir $elixir_version_only-otp-26
elif has erlang@25; then
  install_elixir $elixir_version_only-otp-25
elif has erlang@24; then
  install_elixir $elixir_version_only-otp-24
fi

My other option is to make one rtx plugin that manages both dependencies, but that has the downside of not being able to really reuse Erlang installations (well, it could, but it would be a bit on the messy side).

[jdx]

so the following should work now:

rtx use erlang@26
rtx use elixir@1.15 # erlang will be on PATH during the plugin's `bin/install`

but there is a downside that (at least currently) if you ran rtx use erlang@26 elixir@1.15 it would attempt to install both in parallel and fail.

elixir will have erlang on PATH for installation in this case. I'm not sure I think that we would want to force a relationship where you have to manage erlang with rtx if you manage elixir with rtx. As an example, some users use poetry with rtx but use it with their system or pyenv managed python.

[halostatue]

Hm. I don't think that I agree with your conclusion, having found cases where not remembering to upgrade Erlang leads to weird errors on some Elixir upgrades. Of the languages that I mentioned, only Gleam and Elixir are managed outside of Erlang packaging (LFE and some others are defined as packages within a rebar3 configuration file), but both Gleam and Elixir have instructions indicating "install Erlang, too…".

I wonder if it might not be better to look at making a beam plugin for rtx that you do something like beam@$otp-version[+elixir-$elixir-version][+gleam-$gleam-version] and it knows how to parse those as if they were variants (described by MacPorts).

Having tightly matched build/runtime versions for Elixir is fairly important:

1. Versions built on newer Erlang runtimes cannot run on older Erlang runtimes (you cannot use elixir@1.15-otp-26 if you only have erlang@24 installed).
2. Versions built on older Erlang runtimes can usually run on newer Erlang runtimes, but will usually run with a performance and/or feature penalty due to conditional compilation. There are cases where this is not the case, and some Elixir versions will not run on newer Erlang runtimes than their maximum supported runtime in any case.

I can definitely make a new elixir plug-in that runs both kerl and better Elixir install mechanisms than what is currently present in the asdf-elixir plug-in, but it will mean that multiple versions of Erlang at the same version might be installed rather than taking advantage of the perviously installed erlang@version or indicating that there is no suitable version installed and therefore triggering an install or failing.

[jdx]

multiple versions of Erlang at the same version might be installed rather than taking advantage of the perviously installed

not necessarily. The elixir plugin could run rtx install erlang@... inside bin/install and use an erlang install provided by asdf-erlang.

[jdx]

you could even do something like check $RTX_DATA_DIR/installs/erlang/{26,25,24} to see if any of those symlinks exist to try to find a compatible version and if you don't find one, then run rtx install erlang@26 to create it

[Nezteb]

Today I realized I'd love to have a way to at least specify that certain plugins depend on others during basic commands like mise install. If not in the plugin itself (which I think makes more sense), at least a way to define such dependencies in the local .mise.toml config.

Recently I was trying to build this Elixir/Erlang project: https://github.com/elixir-tools/next-ls/blob/main/.mise.toml

When running mise install, mise attempts to install Elixir before Erlang, which fails because the Elixir install step can't find erlc.

The workaround is to just manually install each language in the correct order:

mise install erlang
mise install elixir

Just an idea. 😅

[eproxus]

There's an additional problem here when running the mise GitHub action. If one specifies a config like this:

[tools]
erlang = "27"
rebar = "latest"

Mise will try to install Rebar before Erlang (which also doesn't make sense in the same way as installing Elixir before Erlang) and CI jobs fail because of this. Is there a way to manually specify these dependencies somehow? Or some workaround?