search

jdx / mise

dev tools, env vars, task runner
https://mise.jdx.dev
MIT License
9.07k stars 245 forks source link

Process/fork bomb when using the experimental Go, NPM, and Cargo backends, locking up the whole OS/computer #2046

Closed jimeh closed 4 months ago

jimeh commented 4 months ago

Describe the bug

I recently switched over all Go, NPM, and Cargo packages I install globally to use Mise's new experimental backends system. My config is here.

On my main machine, this worked fine, but I already had Go, Rust and Node.js installed and available.

On a new machine however, where nothing was installed, running mise ls (among others) prints an endless sea of errors, while spawning as many processes as the OS kernel will allow, consuming all CPU time and memory. Below is a small sample of the error output:

Error sample ``` mise failed to resolve version of cargo:gitu from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install gitu mise failed to resolve version of npm:jsonlint from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of npm:eslint from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of cargo:gitu from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install gitu mise failed to resolve version of npm:jsonlint from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of npm:typescript from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of npm:eslint from ~/.config/mise/config.toml: No such file or directory (os error 2) 33mmise mise failed to resolve version of npm:prettier-pnp from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of cargo:sccache from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install sccachemise failed to resolve version of npm:jsonlint from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of npm:eslint from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of npm:typescript-formatter from ~/.config/mise/config.toml: No such file or directory (os error 2) mise mise failed to resolve version of cargo:gitu from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install gitu failed to resolve version of npm:jsonlint from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of npm:prettier-pnp from ~/.config/mise/config.toml: No such file or directory (os error 2) 33mise failed to resolve version of cargo:cargo-update from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install cargo-update mise failed to resolve version of cargo:gitu from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install gitu mise failed to resolve version of npm:prettier-pnp from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of cargo:bacon from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install bacon mise failed to resolve version of npm:jsonlint from ~/.config/mise/config.toml: No such file or directory (os error 2) mmise failed to resolve version of cargo:sccache from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install sccache mise failed to resolve version of npm:typescript-formatter from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of cargo:bacon from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install bacon mise failed to resolve version of cargo:difftastic from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install difftastic mise failed to resolve version of npm:eslint_d from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of npm:vscode-css-languageserver-bin from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of npm:yaml-language-server from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of cargo:difftastic from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install difftastic mise failed to resolve version of cargo:bat from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install bat mise failed to resolve version of npm:vscode-css-languageserver-bin from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of cargo:sccache from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install sccache mise failed to resolve version of cargo:cargo-audit from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install cargo-audit mise failed to resolve version of npm:localtunnel from ~/.config/mise/config.toml: No such file or directory (os error 2) ailed to resolve version of cargo:difftastic from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install difftastic mise failed to resolve version of cargo:cargo-audit from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install cargo-audit mise failed to resolve version of cargo:gitu from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install gitul cargo-info mise failed to resolve version of cargo:cargo-info from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install cargo-info mise failed to resolve version of npm:jsonlint from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of npm:typescript-language-server from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of cargo:bat from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install bat mise failed to resolve version of npm:vscode-css-languageserver-bin from ~/.config/mise/config.toml: No such file or directory (os error 2)install cargo-audit 33mmise failed to resolve version of npm:prettier-pnp from ~/.config/mise/config.toml: No such file or directory (os error 2) in order to install cargo-info mise failed to resolve version of cargo:gitu from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install gitutall kubectl-watch mise 33mmise failed to resolve version of cargo:bacon from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install bacon mise 33mmise failed to resolve version of cargo:gitu from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install gitu33m failed to resolve version of npm:jsonlint from ~/.config/mise/config.toml: No such file or directory (os error 2)e install it in order to install difftastic mise failed to resolve version of npm:eslint-config-prettier from ~/.config/mise/config.toml: No such file or directory (os error 2)mise failed to resolve version of npm:dockerfile-language-server-nodejs from ~/.config/mise/config.toml: No such file or directory (os error 2)mise failed to resolve version of npm:prettier from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of npm:eslint_d from ~/.config/mise/config.toml: No such file or directory (os error 2)it in order to install hexyl mise failed to resolve version of npm:jsonlint from ~/.config/mise/config.toml: No such file or directory (os error 2)t in order to install gitu mise failed to resolve version of cargo:bacon from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install baconll watchexec-cli 33mmise failed to resolve version of cargo:cargo-quickinstall from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install cargo-quickinstallo resolve version of cargo:jwt-cli from ~/.config/mise/config.tmise failed to resolve version of npm:stylelint from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of cargo:bacon from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install bacon mise failed to resolve version of npm:prettier-pnp from ~/.config/mise/config.toml: No such file or directory (os error 2)order to install bat mise failed to resolve version of cargo:bacon from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install bacon 33mmise failed to resolve version of npm:vscode-css-languageserver-bin from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of cargo:hexyl from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install hexyl mise failed to resolve version of cargo:hexyl from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install hexyl 33mmise failed to resolve version of cargo:bacon from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install baconmise failed to resolve version of cargo:cargo-binstall from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install cargo-binstall mise failed to resolve version of npm:vscode-css-languageserver-bin from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of npm:markdown-it from ~/.config/mise/config.toml: No such file or directory (os error 2) 33mmise 33failed to resolve version of cargo:gitu from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install gitu mmise failed to resolve version of npm:prettier from ~/.config/mise/config.toml: No such file or directory (os error 2) failed to resolve version of npm:jsonlint from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of npm:vscode-css-languageserver-bin from ~/.config/mise/config.toml: No such file or directory (os error 2) 33mmise failed to resolve version of npm:prettier-pnp from ~/.config/mise/config.toml: No such file or directory (os error 2) mise failed to resolve version of cargo:cargo-info from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install cargo-info mise failed to resolve version of cargo:sccache from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install sccache mise failed to resolve version of cargo:dirstat-rs from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install dirstat-rs mise failed to resolve version of npm:eslint from ~/.config/mise/config.toml: No such file or directory (os error 2) 33mise failed to resolve version of npm:prettier-pnp from ~/.config/mise/config.toml: No such file or directory (os error 2) mmise failed to resolve version of cargo:cargo-edit from ~/.config/mise/config.toml: cargo is not installed. Please install it in order to install cargo-edit ```

And here is a screen recording of me reproducing it in an isolated VM running mise ls:

mise ls screen recording https://github.com/jdx/mise/assets/39563/27cbc787-c091-4787-80bd-016f2f1e4fab

And here it is again when running mise doctor, this time with htop not freezing, so you can see some of the processes that are being spawned:

mise doctor screen recording https://github.com/jdx/mise/assets/39563/8d8fbb83-2e62-4c8c-96d8-e9c07d6143ee

To Reproduce

Start a fresh VM (for example Ubumtu 22.04), install mise, then use a ~/.config/mise/config.toml with something like this following:

config.toml ```toml [tools] "cargo:bacon" = "latest" "cargo:bat" = "latest" "cargo:cargo-audit" = "latest" "cargo:cargo-binstall" = "latest" "cargo:cargo-edit" = "latest" "cargo:cargo-info" = "latest" "cargo:cargo-quickinstall" = "latest" "cargo:cargo-update" = "latest" "cargo:difftastic" = "latest" "cargo:dirstat-rs" = "latest" "cargo:eza" = "latest" "cargo:gitu" = "0.19" "cargo:hexyl" = "latest" "cargo:hwatch" = "latest" "cargo:jwt-cli" = "latest" "cargo:kubectl-watch" = "latest" "cargo:sccache" = "latest" "cargo:tlrc" = "latest" "cargo:ubi" = "latest" "cargo:watchexec-cli" = "latest" "go:github.com/bufbuild/buf-language-server/cmd/bufls" = "latest" "go:github.com/go-delve/delve/cmd/dlv" = "latest" "go:github.com/golang/mock/mockgen" = "latest" "go:github.com/hashicorp/terraform-ls" = "latest" "go:github.com/nametake/golangci-lint-langserver" = "latest" "go:github.com/rakyll/hey" = "latest" "go:github.com/segmentio/golines" = "latest" "go:golang.org/x/tools/cmd/godoc" = "latest" "go:golang.org/x/tools/cmd/goimports" = "latest" "go:golang.org/x/tools/gopls" = "latest" "go:golang.org/x/vuln/cmd/govulncheck" = "latest" "go:google.golang.org/grpc/cmd/protoc-gen-go-grpc" = "latest" "go:google.golang.org/protobuf/cmd/protoc-gen-go" = "latest" "go:mvdan.cc/gofumpt" = "latest" "npm:@mermaid-js/mermaid-cli" = "latest" "npm:@prettier/plugin-php" = "latest" "npm:@prettier/plugin-ruby" = "latest" "npm:@prettier/plugin-xml" = "latest" "npm:dockerfile-language-server-nodejs" = "latest" "npm:eslint" = "latest" "npm:eslint-config-prettier" = "latest" "npm:eslint-plugin-prettier" = "latest" "npm:eslint_d" = "latest" "npm:htmllint-cli" = "latest" "npm:httpsnippet" = "latest" "npm:jsonlint" = "latest" "npm:localtunnel" = "latest" "npm:markdown-it" = "latest" "npm:prettier" = "latest" "npm:prettier-plugin-toml" = "latest" "npm:prettier-pnp" = "latest" "npm:stylelint" = "latest" "npm:typescript" = "latest" "npm:typescript-formatter" = "latest" "npm:typescript-language-server" = "latest" "npm:uuid-cli" = "latest" "npm:vscode-css-languageserver-bin" = "latest" "npm:vscode-json-languageserver" = "latest" "npm:yaml-language-server" = "latest" 1password-cli = "latest" actionlint = "latest" buf = "latest" ctop = "latest" direnv = "latest" dust = "latest" github-cli = "latest" go = "latest" golangci-lint = "latest" goreleaser = "latest" hadolint = "latest" helm = "latest" jq = "latest" k9s = "latest" kubectl = "latest" kubectx = "latest" lua = "latest" neovim = "latest" node = "lts" python = "latest" rclone = "latest" ripgrep = "latest" ruby = "latest" shellcheck = "latest" shfmt = "latest" starship = "latest" terraform = "latest" tflint = "latest" usage = "latest" yj = "latest" yq = "latest" zoxide = "latest" ```

Then run mise ls, and it should start printing an endless sea of warnings, while spawing as many processes as your Kernel will allow, eating up all CPU time and system memory.

Expected behavior

mise ls simply prints a list of all tools, with them all marked as missing.

mise doctor output

mise doctor yields the same behavior as mise ls, see screen recording further up.

Additional context

I've observed this behavior on both macOS Sonoma, and Ubuntu 22.04.

jdx commented 4 months ago

Yeah this was reported in another ticket. I added a check to ensure npm/* exists but that check is ending up in a loop, I'm guessing it's shim-related. This is high on my list when I can get around to mise work.

jimeh commented 4 months ago

Ah, yeah the shim cause makes sense. In the mise doctor screen recording, you can see the processes it's running are all go list -m -versions -json <pgk-name>.

But if the go executable it's using is a shim to mise, said invocation of go, must be triggering the same lookup of missing packages again, eventually only failing when the kernel fails to launch the go shim due to too many processes running.