jdx / mise

dev tools, env vars, task runner
https://mise.jdx.dev
MIT License
10.44k stars 301 forks source link

Ruby installs hang when shims and auto-install are enabled #2545

Open jeremy opened 2 months ago

jeremy commented 2 months ago

Describe the bug Ruby builds detect the mise ruby shim and try to invoke it, triggering recursive auto-install of the same version currently being built, causing the build to hang indefinitely.

To Reproduce Set a ruby tool version, activate mise shims, and attempt mise install.

Minimal reproduction, starting with mise not yet activated in the shell.

> mise uninstall ruby@3.3.5
mise ruby@3.3.5 is not installed

> which ruby
/usr/bin/ruby

> cat .mise.toml 
[tools]
ruby = "3.3.5"

> mise doctor
mise doctor
version: 2024.9.0 macos-arm64 (c83103c 2024-09-05)
activated: no
shims_on_path: no

build_info: 
  Target: aarch64-apple-darwin
  Features: DEFAULT, NATIVE_TLS, OPENSSL
  Built: Thu, 5 Sep 2024 00:42:43 +0000
  Rust Version: rustc 1.80.1 (3f5fd8dd4 2024-08-06)
  Profile: release

shell: 
  /opt/homebrew/bin/bash
  GNU bash, version 5.2.32(1)-release (aarch64-apple-darwin23.4.0)
  Copyright (C) 2022 Free Software Foundation, Inc.
  License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

  This is free software; you are free to change and redistribute it.
  There is NO WARRANTY, to the extent permitted by law.

dirs: 
  data: ~/.local/share/mise
  config: ~/.config/mise
  cache: ~/Library/Caches/mise
  state: ~/.local/state/mise
  shims: ~/.local/share/mise/shims

config_files: 
  ~/.config/mise/config.toml
  ~/foo/.mise.toml

backends: 
  cargo
  core
  go
  npm
  pipx
  spm
  ubi
  vfox

plugins: 
  yarn  https://github.com/mise-plugins/asdf-yarn.git#74ea3b9

toolset: 
  ruby@3.3.5    (missing)
  go@1.22.4     
  node@20.15.0  
  yarn@1.22.22  

env_vars: 
  (none)

settings: 
  activate_aggressive = false
  all_compile = false
  always_keep_download = false
  always_keep_install = false
  asdf = true
  asdf_compat = false
  cargo_binstall = true
  color = true
  disable_default_shorthands = false
  disable_hints = []
  disable_tools = []
  experimental = true
  go_default_packages_file = "~/.default-go-packages"
  go_download_mirror = "https://dl.google.com/go"
  go_repo = "https://github.com/golang/go"
  go_set_gopath = false
  go_set_goroot = true
  go_skip_checksum = false
  http_timeout = 30
  jobs = 4
  legacy_version_file = true
  legacy_version_file_disable_tools = []
  libgit2 = true
  node_compile = false
  not_found_auto_install = true
  paranoid = false
  pipx_uvx = false
  plugin_autoupdate_last_check_duration = "7d"
  python_default_packages_file = "~/.default-python-packages"
  python_pyenv_repo = "https://github.com/pyenv/pyenv.git"
  raw = false
  trusted_config_paths = []
  quiet = false
  use_versions_host = true
  verbose = false
  vfox = false
  yes = false
  ci = false
  debug = false
  trace = false
  log_level = "info"
  python_venv_auto_create = false

  [status]
  missing_tools = "if_other_versions_installed"
  show_env = false
  show_tools = false

No warnings found
1 problem found:

1. mise is not activated, run mise help activate or
   read documentation at https://mise.jdx.dev for activation instructions.
   Alternatively, add the shims directory ~/.local/share/mise/shims to PATH.
   Using the shims directory is preferred for non-interactive setups.

Activate shims and install Ruby:

> OLDPATH="$PATH"
> eval "$(~/.local/bin/mise activate bash --shims)"
> which ruby
/Users/jeremy/.local/share/mise/shims/ruby
> MISE_DEBUG=1 mise install
[DEBUG] ARGS: mise install
[DEBUG] $ /Users/jeremy/.local/share/mise/plugins/yarn/bin/list-legacy-filenames 
[DEBUG] Config {
    Config Files: [
        "~/foo/.mise.toml",
        "~/.config/mise/config.toml",
    ],
}
[DEBUG] ToolRequestSet.build(831.459µs): ToolRequestSet: ruby@3.3.5 go@latest node@lts yarn@1

[DEBUG] install_versions: ruby@3.3.5
[DEBUG] $ /Users/jeremy/Library/Caches/mise/ruby/ruby-build/bin/ruby-build --version
[DEBUG] GET https://api.github.com/repos/rbenv/ruby-build/releases/latest
[DEBUG] starting new connection: https://api.github.com/
[DEBUG] GET https://api.github.com/repos/rbenv/ruby-build/releases/latest 200 OK
running ruby-build
[DEBUG] $ ~/Library/Caches/mise/ruby/ruby-build/bin/ruby-build --verbose 3.3.5 /Users/jeremy/.local/share/mise/installs/ruby/3.3.5
ruby-build: using openssl@3 from homebrew
cd /var/folders/w7/bscl5y995xjgg740rb2s79yw0000gn/T/ruby-build.20240909111651.22855.9siWRk
==> Downloading ruby-3.3.5.tar.gz...
-> curl -q -fL -o ruby-3.3.5.tar.gz https://cache.ruby-lang.org/pub/ruby/3.3/ruby-3.3.5.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 21.1M  100 21.1M    0     0  36.3M      0 --:--:-- --:--:-- --:--:-- 36.3M
cd /var/folders/w7/bscl5y995xjgg740rb2s79yw0000gn/T/ruby-build.20240909111651.22855.9siWRk/ruby-3.3.5
==> Installing ruby-3.3.5...
ruby-build: using libyaml from homebrew
ruby-build: using gmp from homebrew
-> ./configure "--prefix=$HOME/.local/share/mise/installs/ruby/3.3.5" --with-openssl-dir=/opt/homebrew/opt/openssl@3 --enable-shared --with-libyaml-dir=/opt/homebrew/opt/libyaml --with-gmp-dir=/opt/homebrew/opt/gmp --with-ext=openssl,psych,+
checking for ruby... /Users/jeremy/.local/share/mise/shims/ruby
^C
# Hangs here, attempting recursively auto-install?**

Remove shims from PATH to avoid inadvertent recursive install:

> PATH="$OLDPATH"
> MISE_DEBUG=1 mise install
[DEBUG] ARGS: mise install
[DEBUG] Config {
    Config Files: [
        "~/foo/.mise.toml",
        "~/.config/mise/config.toml",
    ],
}
[DEBUG] ToolRequestSet.build(103.209µs): ToolRequestSet: ruby@3.3.5 go@latest node@lts yarn@1

[DEBUG] install_versions: ruby@3.3.5
[DEBUG] $ /Users/jeremy/Library/Caches/mise/ruby/ruby-build/bin/ruby-build --version
[DEBUG] GET https://api.github.com/repos/rbenv/ruby-build/releases/latest
[DEBUG] starting new connection: https://api.github.com/
[DEBUG] GET https://api.github.com/repos/rbenv/ruby-build/releases/latest 200 OK
running ruby-build
[DEBUG] $ ~/Library/Caches/mise/ruby/ruby-build/bin/ruby-build --verbose 3.3.5 /Users/jeremy/.local/share/mise/installs/ruby/3.3.5
ruby-build: using openssl@3 from homebrew
cd /var/folders/w7/bscl5y995xjgg740rb2s79yw0000gn/T/ruby-build.20240909111744.23685.oiVevH
==> Downloading ruby-3.3.5.tar.gz...
-> curl -q -fL -o ruby-3.3.5.tar.gz https://cache.ruby-lang.org/pub/ruby/3.3/ruby-3.3.5.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 21.1M  100 21.1M    0     0  44.8M      0 --:--:-- --:--:-- --:--:-- 44.9M
cd /var/folders/w7/bscl5y995xjgg740rb2s79yw0000gn/T/ruby-build.20240909111744.23685.oiVevH/ruby-3.3.5
==> Installing ruby-3.3.5...
ruby-build: using libyaml from homebrew
ruby-build: using gmp from homebrew
-> ./configure "--prefix=$HOME/.local/share/mise/installs/ruby/3.3.5" --with-openssl-dir=/opt/homebrew/opt/openssl@3 --enable-shared --with-libyaml-dir=/opt/homebrew/opt/libyaml --with-gmp-dir=/opt/homebrew/opt/gmp --with-ext=openssl,psych,+
checking for ruby... /usr/bin/ruby
…
**# Ruby build proceeds successfully from here**

Or use MISE_NOT_FOUND_AUTO_INSTALL=0 to prevent the shim from initiating a recursive install:

[DEBUG] ARGS: mise install
[DEBUG] Config {
    Config Files: [
        "~/foo/.mise.toml",
        "~/.config/mise/config.toml",
    ],
}
[DEBUG] ToolRequestSet.build(718.875µs): ToolRequestSet: ruby@3.3.5 go@latest node@lts yarn@1

[DEBUG] install_versions: ruby@3.3.5
[DEBUG] $ /Users/jeremy/Library/Caches/mise/ruby/ruby-build/bin/ruby-build --version
[DEBUG] GET https://api.github.com/repos/rbenv/ruby-build/releases/latest
[DEBUG] starting new connection: https://api.github.com/
[DEBUG] GET https://api.github.com/repos/rbenv/ruby-build/releases/latest 200 OK
running ruby-build
[DEBUG] $ ~/Library/Caches/mise/ruby/ruby-build/bin/ruby-build --verbose 3.3.5 /Users/jeremy/.local/share/mise/installs/ruby/3.3.5
ruby-build: using openssl@3 from homebrew
cd /var/folders/w7/bscl5y995xjgg740rb2s79yw0000gn/T/ruby-build.20240909115029.44770.kc7eqX
==> Downloading ruby-3.3.5.tar.gz...
-> curl -q -fL -o ruby-3.3.5.tar.gz https://cache.ruby-lang.org/pub/ruby/3.3/ruby-3.3.5.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 21.1M  100 21.1M    0     0  44.9M      0 --:--:-- --:--:-- --:--:-- 44.9M
cd /var/folders/w7/bscl5y995xjgg740rb2s79yw0000gn/T/ruby-build.20240909115029.44770.kc7eqX/ruby-3.3.5
==> Installing ruby-3.3.5...
ruby-build: using libyaml from homebrew
ruby-build: using gmp from homebrew
-> ./configure "--prefix=$HOME/.local/share/mise/installs/ruby/3.3.5" --with-openssl-dir=/opt/homebrew/opt/openssl@3 --enable-shared --with-libyaml-dir=/opt/homebrew/opt/libyaml --with-gmp-dir=/opt/homebrew/opt/gmp --with-ext=openssl,psych,+
**checking for ruby... /Users/jeremy/.local/share/mise/shims/ruby
# Shim found in path, but auto-install is not triggered**
[DEBUG] Config {
    Config Files: [
        "~/.config/mise/config.toml",
    ],
}
[DEBUG] Toolset (367.791µs): ruby@3.3, go@latest, node@lts, yarn@1
[DEBUG] Toolset (282.166µs): ruby@3.3, go@latest, node@lts, yarn@1
[DEBUG] Config {
    Config Files: [
        "~/.config/mise/config.toml",
    ],
}
[DEBUG] Toolset (375.375µs): ruby@3.3, go@latest, node@lts, yarn@1
[DEBUG] Toolset (222.916µs): ruby@3.3, go@latest, node@lts, yarn@1
tool/config.guess already exists
tool/config.sub already exists
…

From users' point of view, it looks like Ruby is building but taking forever, when it fact it indefinitely hangs when the configure script invokes the ruby shim.

roele commented 2 months ago

I am struggling to reproduce this. What i find odd is that your first mise doctor command does not report ruby as (missing). So it already is installed (via Homebrew)?

jeremy commented 2 months ago

Ah it was (missing). I must have copy/pasted the repro out of order. Updated. (And confirmed again.)

jeremy commented 2 months ago

Curious how your output from the above commands compares.

Key is that the Ruby configure script looks for an existing Ruby, finds the shim, invokes it, which triggers auto-install (and presumably hits a lock or something, causing the hang).

When shims are absent or auto-install is disabled, the Ruby build can't tickle that code path.

fredwu commented 2 months ago

Just chiming in to say that I just ran into this exact issue. Had to disable shim for ruby to compile/install.

jeremy commented 2 months ago

Running with CONFIG_SHELL="bash -x" to get debug output from the Ruby configure script makes it clear the hang is when it tries to invoke the ruby shim for the first time:

> MISE_TRACE=1 CONFIG_SHELL="bash -x" mise install
[DEBUG] (1) mise::cli: [src/cli/mod.rs:219] ARGS: mise install
[TRACE] (1) mise::config: [src/config/mod.rs:69] Settings: Settings {
    activate_aggressive: false,
    all_compile: false,
    always_keep_download: false,
    always_keep_install: false,
    asdf: true,
    asdf_compat: false,
    cargo_binstall: true,
    color: true,
    disable_default_shorthands: false,
    disable_hints: {},
    disable_tools: {},
    experimental: true,
    go_default_packages_file: "~/.default-go-packages",
    go_download_mirror: "https://dl.google.com/go",
    go_repo: "https://github.com/golang/go",
    go_set_gobin: None,
    go_set_gopath: false,
    go_set_goroot: true,
    go_skip_checksum: false,
    http_timeout: 30,
    jobs: 4,
    legacy_version_file: true,
    legacy_version_file_disable_tools: {},
    libgit2: true,
    node_compile: false,
    not_found_auto_install: true,
    paranoid: false,
    pipx_uvx: false,
    plugin_autoupdate_last_check_duration: "7d",
    python_compile: None,
    python_default_packages_file: Some(
        "/Users/jeremy/.default-python-packages",
    ),
    python_patch_url: None,
    python_patches_directory: None,
    python_precompiled_arch: None,
    python_precompiled_os: None,
    python_pyenv_repo: "https://github.com/pyenv/pyenv.git",
    raw: false,
    shorthands_file: None,
    status: SettingsStatus {
        missing_tools: IfOtherVersionsInstalled,
        show_env: false,
        show_tools: false,
    },
    task_output: None,
    trusted_config_paths: {},
    quiet: false,
    use_versions_host: true,
    verbose: true,
    vfox: false,
    yes: false,
    ci: false,
    cd: None,
    debug: true,
    env_file: None,
    trace: true,
    log_level: "trace",
    python_venv_auto_create: false,
}
[TRACE] (2) mise::file: [src/file.rs:130] cat ~/.local/share/mise/installs/go/.mise.backend.json
[TRACE] (5) mise::file: [src/file.rs:130] cat ~/.local/share/mise/installs/yarn/.mise.backend.json
[TRACE] (6) mise::file: [src/file.rs:130] cat ~/.local/share/mise/installs/ruby/.mise.backend.json
[TRACE] (4) mise::file: [src/file.rs:130] cat ~/.local/share/mise/installs/node/.mise.backend.json
[TRACE] (7) mise::cache: [src/cache.rs:87] reading ~/Library/Caches/mise/yarn/legacy_filenames-9f9e5.msgpack.z
[TRACE] (2) mise::config::config_file::mise_toml: [src/config/config_file/mise_toml.rs:85] parsing: ~/foo/.mise.toml
[TRACE] (2) mise::file: [src/file.rs:130] cat ~/foo/.mise.toml
[TRACE] (3) mise::config::config_file::mise_toml: [src/config/config_file/mise_toml.rs:85] parsing: ~/.config/mise/config.toml
[TRACE] (3) mise::file: [src/file.rs:130] cat ~/.config/mise/config.toml
[TRACE] (3) mise::file: [src/file.rs:130] cat ~/.config/mise/config.toml
[TRACE] (2) mise::file: [src/file.rs:130] cat ~/foo/.mise.toml
[TRACE] (3) mise::config::config_file::mise_toml: [src/config/config_file/mise_toml.rs:98] [settings]
experimental = true

[tools]
ruby = "3.3"
go = "latest"
node = "lts"
yarn = "1"

[TRACE] (2) mise::config::config_file::mise_toml: [src/config/config_file/mise_toml.rs:98] [tools]
ruby = "3.3.5"

[DEBUG] (1) mise::config: [src/config/mod.rs:90] Config {
    Config Files: [
        "~/foo/.mise.toml",
        "~/.config/mise/config.toml",
    ],
}
[DEBUG] (1) mise::toolset::tool_request_set: [src/toolset/tool_request_set.rs:152] ToolRequestSet.build(58.041µs): ToolRequestSet: ruby@3.3.5 go@latest node@lts yarn@1

[DEBUG] (1) mise::toolset: [src/toolset/mod.rs:163] install_versions: ruby@3.3.5
[TRACE] (10) mise::backend: [src/backend/mod.rs:322] Ensuring dependencies installed for ruby
[TRACE] (10) mise::backend: [src/backend/mod.rs:191] Listing remote versions for ruby
[TRACE] (10) mise::cache: [src/cache.rs:87] reading ~/Library/Caches/mise/ruby/remote_versions-9f9e5.msgpack.z
[TRACE] (10) mise::file: [src/file.rs:32] rm -rf ~/.local/share/mise/downloads/ruby/3.3.5
[TRACE] (10) mise::file: [src/file.rs:32] rm -rf ~/Library/Caches/mise/ruby/3.3.5
[TRACE] (10) mise::file: [src/file.rs:43] rm ~/.local/share/mise/installs/ruby/3.3.5
[TRACE] (10) mise::file: [src/file.rs:149] mkdir -p ~/.local/share/mise/installs/ruby/3.3.5
[TRACE] (10) mise::file: [src/file.rs:149] mkdir -p ~/.local/share/mise/downloads/ruby/3.3.5
[TRACE] (10) mise::file: [src/file.rs:149] mkdir -p ~/Library/Caches/mise/ruby/3.3.5
[DEBUG] (10) mise::cmd: [src/cmd.rs:91] $ /Users/jeremy/Library/Caches/mise/ruby/ruby-build/bin/ruby-build --version
[DEBUG] (10) mise::http: [src/http.rs:52] GET https://api.github.com/repos/rbenv/ruby-build/releases/latest
[DEBUG] (10) reqwest::connect: [/Users/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/reqwest-0.12.7/src/connect.rs:497] starting new connection: https://api.github.com/
[DEBUG] (10) mise::http: [src/http.rs:60] GET https://api.github.com/repos/rbenv/ruby-build/releases/latest 200 OK
running ruby-build
[DEBUG] (10) mise::cmd: [src/cmd.rs:244] $ ~/Library/Caches/mise/ruby/ruby-build/bin/ruby-build --verbose 3.3.5 /Users/jeremy/.local/share/mise/installs/ruby/3.3.5
ruby-build: using openssl@3 from homebrew
cd /var/folders/w7/bscl5y995xjgg740rb2s79yw0000gn/T/ruby-build.20240909233924.4504.hT4cI3
==> Downloading ruby-3.3.5.tar.gz...
-> curl -q -fL -o ruby-3.3.5.tar.gz https://cache.ruby-lang.org/pub/ruby/3.3/ruby-3.3.5.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 21.1M  100 21.1M    0     0  29.6M      0 --:--:-- --:--:-- --:--:-- 29.5M
cd /var/folders/w7/bscl5y995xjgg740rb2s79yw0000gn/T/ruby-build.20240909233924.4504.hT4cI3/ruby-3.3.5
==> Installing ruby-3.3.5...
ruby-build: using libyaml from homebrew
ruby-build: using gmp from homebrew
-> ./configure "--prefix=$HOME/.local/share/mise/installs/ruby/3.3.5" --with-openssl-dir=/opt/homebrew/opt/openssl@3 --enable-shared --with-libyaml-dir=/opt/homebrew/opt/libyaml --with-gmp-dir=/opt/homebrew/opt/gmp --with-ext=openssl,psych,+
# (omitting configure noise)
+ begin_group 'environment section'
+ :
+ HAVE_BASERUBY=yes
+ BASERUBY_VERSION=
+ test
+ as_fn_nop
+ return 1
+ set dummy ruby
+ ac_word=ruby
+ printf '%s\n' 'configure:3705: checking for ruby'
+ printf %s 'checking for ruby... '
+ test
+ as_fn_nop
+ return 1
+ case $BASERUBY in
+ as_save_IFS='     
'
+ IFS=:
+ for as_dir in $PATH
+ IFS='     
'
+ case $as_dir in
+ as_dir=/Users/jeremy/.local/share/mise/shims/
+ for ac_exec_ext in '' $ac_executable_extensions
+ as_fn_executable_p /Users/jeremy/.local/share/mise/shims/ruby
+ test -f /Users/jeremy/.local/share/mise/shims/ruby
+ test -x /Users/jeremy/.local/share/mise/shims/ruby
+ ac_cv_path_BASERUBY=/Users/jeremy/.local/share/mise/shims/ruby
+ printf '%s\n' 'configure:3728: found /Users/jeremy/.local/share/mise/shims/ruby'
+ break 2
+ IFS='     
'
+ test -z /Users/jeremy/.local/share/mise/shims/ruby
+ BASERUBY=/Users/jeremy/.local/share/mise/shims/ruby
+ test -n /Users/jeremy/.local/share/mise/shims/ruby
+ printf '%s\n' 'configure:3741: result: /Users/jeremy/.local/share/mise/shims/ruby'
+ colorize_result /Users/jeremy/.local/share/mise/shims/ruby
checking for ruby... /Users/jeremy/.local/share/mise/shims/ruby
+ case "$1" in
+ :
+ printf '%s\n' /Users/jeremy/.local/share/mise/shims/ruby
++ RUBYOPT=-
++ /Users/jeremy/.local/share/mise/shims/ruby --disable=gems -e 'print 42 if RUBY_VERSION > "2.5"'
^C

Interestingly, no MISE_TRACE output from the shim invocation.

jacobjuul commented 2 months ago

Having the same issue currently

jacobjuul commented 2 months ago
> mise uninstall ruby --all
> mise reshim

seemed to work for me