Ruby build not defaulting to using the Mise host to fetch versions

[lucasteligioridis]

Describe the bug Looking at the documentation over here: https://mise.jdx.dev/configuration.html#mise-use-versions-host

Describes that this setting is default and should use the mise host instead of Github to fetch the current versions list and understand what to download.

Locally I am testing this and not seeing this behaviour, its constantly going to Githubs API to fetch the available versions. Reason this has come up is because we have been getting rate limited on the API and this seems to be the cause.

I looked into the code base to try and understand deeper and can only see this path being the only function that does not early return or use the fetch_remote_versions_from_mise function, so would be going to Github by default: https://github.com/jdx/mise/blob/f8ce139c1d0b41006dbbf1707801bf665f201ec6/src/plugins/core/ruby.rs#L355

I am not a Rust developer so this is as far as my skillset will allow me to debug further unfortunately.

I even tried to override the environment variable MISE_USE_VERSIONS_HOST=true to see if the default wasn't working, but this had the same result.

To Reproduce Install a version of ruby on your machine with --verbose mode and see mise fetching the version from Github instead of the mise URL.

λ mise install --verbose
[DEBUG] ARGS: /opt/homebrew/bin/mise install --verbose
[DEBUG] Config {
    Config Files: [
        "~/.mise.toml",
    ],
    Env: {
        "MISE_USE_VERSIONS_HOST": "true",
    },
}
[DEBUG] Toolset: postgres@15.4, redis@7.0.14, python@3.11.7, gcloud@460.0.0, yarn@1.22.19, node@20.5.1, ruby@3.2.4, java@temurin-17.0.0+35, kubectl@1.27.4
[DEBUG] $ /Users/lucasteligiordis/Library/Caches/mise/ruby/ruby-build/bin/ruby-build --version
[DEBUG] GET https://api.github.com/repos/rbenv/ruby-build/releases/latest
[DEBUG] starting new connection: https://api.github.com/
[DEBUG] GET https://api.github.com/repos/rbenv/ruby-build/releases/latest 200 OK
running ruby-build

Expected behavior Use the mise host instead of Github to prevent being rate limited.

mise doctor output

version: 2024.4.12 macos-arm64 (2024-04-30)
activated: yes
shims_on_path: no

build_info:
  Target: aarch64-apple-darwin
  Features: DEFAULT, NATIVE_TLS
  Built: Tue, 30 Apr 2024 23:30:07 +0000
  Rust Version: rustc 1.77.2 (25ef9e3d8 2024-04-09) (Homebrew)
  Profile: release

shell:
  /opt/homebrew/bin/bash
  GNU bash, version 5.2.26(1)-release (aarch64-apple-darwin23.2.0)
  Copyright (C) 2022 Free Software Foundation, Inc.
  License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

  This is free software; you are free to change and redistribute it.
  There is NO WARRANTY, to the extent permitted by law.

dirs:
  data: ~/.config/mise
  config: ~/.config/mise
  cache: ~/Library/Caches/mise
  state: ~/.local/state/mise
  shims: ~/.config/mise/shims

config_files:
  ~/.mise.toml

backends:
  cargo
  go
  npm
  pipx
  ubi

plugins:
  bun        (core)
  deno       (core)
  gcloud     https://github.com/jthegedus/asdf-gcloud.git#00cdf06
  go         (core)
  java       (core)
  kubectl    https://github.com/asdf-community/asdf-kubectl.git#cbe6df4
  node       (core)
  packer     https://github.com/asdf-community/asdf-hashicorp.git#197e3ec
  postgres   https://github.com/smashedtoatoms/asdf-postgres.git#4592b64
  python     (core)
  redis      https://github.com/smashedtoatoms/asdf-redis.git#d5a67c5
  ruby       (core)
  terraform  https://github.com/asdf-community/asdf-hashicorp.git#197e3ec
  yarn       https://github.com/twuni/asdf-yarn.git#376c540

toolset:
  postgres@15.4
  redis@7.0.14
  python@3.11.7
  gcloud@460.0.0
  yarn@1.22.19
  node@20.5.1
  ruby@3.2.4            (missing)
  java@temurin-17.0.0+35
  kubectl@1.27.4

env_vars:
  MISE_USE_VERSIONS_HOST=true
  MISE_SHELL=bash

settings:
  activate_aggressive = true
  all_compile = false
  always_keep_download = false
  always_keep_install = false
  asdf_compat = false
  cargo_binstall = true
  color = true
  disable_default_shorthands = false
  disable_tools = []
  experimental = false
  go_default_packages_file = "~/.default-go-packages"
  go_download_mirror = "https://dl.google.com/go"
  go_repo = "https://github.com/golang/go"
  go_set_gopath = false
  go_set_goroot = true
  go_skip_checksum = false
  jobs = 4
  legacy_version_file = true
  legacy_version_file_disable_tools = []
  node_compile = false
  not_found_auto_install = true
  paranoid = false
  plugin_autoupdate_last_check_duration = "7d"
  python_compile = false
  python_default_packages_file = "/Users/lucasteligiordis/.default-python-packages"
  python_pyenv_repo = "https://github.com/pyenv/pyenv.git"
  raw = false
  trusted_config_paths = []
  quiet = false
  verbose = false
  yes = true
  ci = false
  debug = false
  trace = false
  log_level = "info"
  python_venv_auto_create = false

  [status]
  missing_tools = "if_other_versions_installed"
  show_env = false
  show_tools = false

No warnings found
No problems found

Additional context This might be intended behaviour here in which case, any suggestions are welcomed!

I know I can fall back to using the asdf plugin which just has the build version hard coded and does a git fetch instead of using the API at all, but would rather keep the core plugin here and see what can be done before doing that. Appreciate any help! :)

[jdx]

It uses the version host to check for new ruby versions but what you're seeing is it checking for new versions of ruby-build which the versions host does not track.

[lucasteligioridis]

Is there anyway we can force this to use non-github requests here somehow? The rate limit kills us. If I can't change this I'll have to go back to using the asdf ruby plugin in mise, but I'd rather use the core ruby plugin here.

[jdx]

it just shouldn't crash if it can't detect the ruby-build version

[lucasteligioridis]

Crash? Nothing is crashing, everything is working as expected. Its just when we have mise running on our CI hosts, we get smashed with the rate limit due to the huge concurrency so its a bit of a deal breaker for that network.

Just has a large impact.

[jdx]

in this case ruby-build is already installed, it just is throwing an error not allowing anything to proceed because it can't check if that version isn't the latest. In that case we can probably still use ruby-build just fine and should.

[jdx]

in other words, this is an error that should just be a warning

[lucasteligioridis]

Does this code change you just made mean that calls to Github are only made when necessary?

Not sure if there is a bug, but it looks like somewhere in our chain even though we have ruby updated and ruby build installed, Github calls are being made constantly. (we check the ruby version on practically every CI run to ensure it has the right version installed)

[lucasteligioridis]

Ah I see the code change won't make a difference with calls to Github as the call to latest_ruby_build_version is made unconditionally to check the latest version against the installed one.

The core of the issue I raised is to see how we can avoid that call to Github entirely, without caching the version list like mise does with all the other core versions, is there a way I can force mise to not look for a latest release anywhere with an env var or something? Pretty much if I have a version of ruby-build installed I am happy to use that until I deem it necessary to update.

I'd prefer to have a cached version on the mise endpoint like the others and have it auto-update but don't know what the effort that is to get ruby-build there as well.

[jdx]

what is it you're running exactly that is causing this check? it should only be checking the ruby-build version if it is about to actually install a new version of ruby-build or if it's configured not to use the versions host.

Another solution is you could create a github token (doesn't need any roles) and assign it to GITHUB_API_TOKEN which will get around the rate limit problem and that's a convention used throughout both mise and asdf.

[lucasteligioridis]

Here is an output to the API on the network we have this mise ruby install happening:

x-github-media-type: github.v3; format=json
x-ratelimit-limit: 60
x-ratelimit-remaining: 0
x-ratelimit-reset: 1714525541
x-ratelimit-resource: core
x-ratelimit-used: 60
content-length: 280

Even though ruby is already installed, in CI we run a mise install ruby to ensure CI is kept up to sync with the Git repo. 99/100 its a no-op as our ruby version rarely changes.

But I guess somewhere in the code there is still a call being made somewhere to the Github API even though there is actually no ruby to install? So then we get rate limited etc... and engineers (non CI) is impacted when they need to have ruby installed on their machines with the rate limit error. (might be a fresh machine)

[jdx]

anything using that ip address may be hitting github, how do you know it's mise?

[jdx]

when I run mise install ruby it definitely doesn't connect to the github api:

~ ❯ rm -rf ~/.mise/cache/*
~ ❯ mise install ruby --verbose
[DEBUG] ARGS: mise install ruby --verbose
[DEBUG] $ /Users/jdx/.local/share/mise/plugins/tiny/bin/list-legacy-filenames 
[DEBUG] Config {
    Config Files: [
        "~/.config/mise/config.toml",
        "~/.tool-versions",
    ],
}
[DEBUG] $ /Users/jdx/.local/share/mise/plugins/tiny/bin/list-aliases 
[DEBUG] Toolset: node@20.11.0, tiny@3, ruby@latest, python@latest, go@latest, java@22
[DEBUG] GET http://mise-versions.jdx.dev/ruby
[DEBUG] starting new connection: http://mise-versions.jdx.dev/
[DEBUG] GET http://mise-versions.jdx.dev/ruby 200 OK

[lucasteligioridis]

Because I just migrated mise to the machines and replaced it from asdf and thats when we started seeing the issue.

Also your output above is exactly what I experienced locally as well (on my machine). So perhaps it was still catching up over the migration and doing the initial load of ruby.

This still means without the GITHUB_API_TOKEN I'll get rate limited whenever we do a ruby update/install.

[jdx]

yes, but it shouldn't cause a real problem. Unless you're connecting to github for other things, but then those other things would probably be hitting rate limits anyways.

[jdx]

in the asdf plugin this isn't a problem because they hardcode the ruby-build version—the downside of that is anytime a new version of ruby comes out you have to update the plugin before you will see the new version of ruby. If you want that behavior though you can just install asdf-ruby into mise.

[lucasteligioridis]

The real problem is that unfortunately these CI machines run from our "main" office, which is under a single public IP. So when these machines do a ruby update due to a new version that we've likely bumped, means all our engineers will need this same update, anyone in the office will be rate limited and not able to install ruby etc...

That was the actual problem that we faced.

[jdx]

anyone in the office will be rate limited and not able to install ruby etc

but this is what my patch fixes

[lucasteligioridis]

you can just install asdf-ruby into mise

totally get it, looked at their source code and saw that its hard coded, just wanting to initially avoid that as I'd rather support the core mise plugins initially.

[jdx]

thinking it through just now, I suppose there is an edge case that I can sort out. If it can't detect the ruby-build version right now it just uses the old one but it's probably better to just attempt to update if it can't get the version

[lucasteligioridis]

but this is what my patch fixes

Yeah so the patch would mean that if we have people who have bootstrapped already and have a version of ruby-build then it wouldn't block them and it would carry on right? Unless its a new machine entirely.

[jdx]

then it wouldn't block them and it would carry on right

yes, with the old version of ruby-build, but I'm going to change this slightly to attempt to update anyways

Unless its a new machine entirely.

new machines don't need to check the version, it's assumed to be out of date

[lucasteligioridis]

new machines don't need to check the version, it's assumed to be out of date

Oh right, okay so maybe now that we're all up and running and updated and everything is migrated over and there is no mass onboarding of it. In theory we should have reduced hits to Github's API and even a new machine shouldn't be impacted?

[lucasteligioridis]

slightly to attempt to update anyways

No worries, happy to test out as well if you want some test-cases.

[jdx]

while you wait for the next release you could delete ~/.cache/mise/ruby/ruby-build before installing ruby which would prevent it from attempting to check the version

[lucasteligioridis]

Okay thats not a bad workaround! Can you TL;DR me in that case how it understands what version to even install without hitting the Github API to check what to download?

[jdx]

lol: https://github.com/jdx/mise/blob/main/src/plugins/core/ruby.rs#L108-L109

lmk if you don't get it, it just git clone's from master every time

[lucasteligioridis]

Oh haha, yeah thats obvious now that I think about it 🙃