Closed lucasteligioridis closed 6 months ago
It uses the version host to check for new ruby versions but what you're seeing is it checking for new versions of ruby-build which the versions host does not track.
Is there anyway we can force this to use non-github requests here somehow? The rate limit kills us.
If I can't change this I'll have to go back to using the asdf
ruby plugin in mise
, but I'd rather use the core
ruby plugin here.
it just shouldn't crash if it can't detect the ruby-build version
Crash? Nothing is crashing, everything is working as expected. Its just when we have mise
running on our CI hosts, we get smashed with the rate limit due to the huge concurrency so its a bit of a deal breaker for that network.
Just has a large impact.
in this case ruby-build is already installed, it just is throwing an error not allowing anything to proceed because it can't check if that version isn't the latest. In that case we can probably still use ruby-build just fine and should.
in other words, this is an error that should just be a warning
Does this code change you just made mean that calls to Github are only made when necessary?
Not sure if there is a bug, but it looks like somewhere in our chain even though we have ruby updated and ruby build installed, Github calls are being made constantly. (we check the ruby version on practically every CI run to ensure it has the right version installed)
Ah I see the code change won't make a difference with calls to Github as the call to latest_ruby_build_version
is made unconditionally to check the latest
version against the installed one.
The core of the issue I raised is to see how we can avoid that call to Github entirely, without caching the version list like mise
does with all the other core versions, is there a way I can force mise
to not look for a latest
release anywhere with an env var or something? Pretty much if I have a version of ruby-build
installed I am happy to use that until I deem it necessary to update.
I'd prefer to have a cached version on the mise
endpoint like the others and have it auto-update but don't know what the effort that is to get ruby-build there as well.
what is it you're running exactly that is causing this check? it should only be checking the ruby-build version if it is about to actually install a new version of ruby-build or if it's configured not to use the versions host.
Another solution is you could create a github token (doesn't need any roles) and assign it to GITHUB_API_TOKEN which will get around the rate limit problem and that's a convention used throughout both mise and asdf.
Here is an output to the API on the network we have this mise ruby install happening:
x-github-media-type: github.v3; format=json
x-ratelimit-limit: 60
x-ratelimit-remaining: 0
x-ratelimit-reset: 1714525541
x-ratelimit-resource: core
x-ratelimit-used: 60
content-length: 280
Even though ruby is already installed, in CI we run a mise install ruby
to ensure CI is kept up to sync with the Git repo. 99/100 its a no-op as our ruby version rarely changes.
But I guess somewhere in the code there is still a call being made somewhere to the Github API even though there is actually no ruby to install? So then we get rate limited etc... and engineers (non CI) is impacted when they need to have ruby installed on their machines with the rate limit error. (might be a fresh machine)
anything using that ip address may be hitting github, how do you know it's mise?
when I run mise install ruby
it definitely doesn't connect to the github api:
~ ❯ rm -rf ~/.mise/cache/*
~ ❯ mise install ruby --verbose
[DEBUG] ARGS: mise install ruby --verbose
[DEBUG] $ /Users/jdx/.local/share/mise/plugins/tiny/bin/list-legacy-filenames
[DEBUG] Config {
Config Files: [
"~/.config/mise/config.toml",
"~/.tool-versions",
],
}
[DEBUG] $ /Users/jdx/.local/share/mise/plugins/tiny/bin/list-aliases
[DEBUG] Toolset: node@20.11.0, tiny@3, ruby@latest, python@latest, go@latest, java@22
[DEBUG] GET http://mise-versions.jdx.dev/ruby
[DEBUG] starting new connection: http://mise-versions.jdx.dev/
[DEBUG] GET http://mise-versions.jdx.dev/ruby 200 OK
Because I just migrated mise
to the machines and replaced it from asdf
and thats when we started seeing the issue.
Also your output above is exactly what I experienced locally as well (on my machine). So perhaps it was still catching up over the migration and doing the initial load of ruby
.
This still means without the GITHUB_API_TOKEN
I'll get rate limited whenever we do a ruby update/install.
yes, but it shouldn't cause a real problem. Unless you're connecting to github for other things, but then those other things would probably be hitting rate limits anyways.
in the asdf plugin this isn't a problem because they hardcode the ruby-build version—the downside of that is anytime a new version of ruby comes out you have to update the plugin before you will see the new version of ruby. If you want that behavior though you can just install asdf-ruby into mise.
The real problem is that unfortunately these CI machines run from our "main" office, which is under a single public IP.
So when these machines do a ruby update due to a new version that we've likely bumped, means all our engineers will need this same update, anyone in the office will be rate limited and not able to install ruby
etc...
That was the actual problem that we faced.
anyone in the office will be rate limited and not able to install ruby etc
but this is what my patch fixes
you can just install asdf-ruby into mise
totally get it, looked at their source code and saw that its hard coded, just wanting to initially avoid that as I'd rather support the core mise
plugins initially.
thinking it through just now, I suppose there is an edge case that I can sort out. If it can't detect the ruby-build version right now it just uses the old one but it's probably better to just attempt to update if it can't get the version
but this is what my patch fixes
Yeah so the patch would mean that if we have people who have bootstrapped already and have a version of ruby-build
then it wouldn't block them and it would carry on right? Unless its a new machine entirely.
then it wouldn't block them and it would carry on right
yes, with the old version of ruby-build, but I'm going to change this slightly to attempt to update anyways
Unless its a new machine entirely.
new machines don't need to check the version, it's assumed to be out of date
new machines don't need to check the version, it's assumed to be out of date
Oh right, okay so maybe now that we're all up and running and updated and everything is migrated over and there is no mass onboarding of it. In theory we should have reduced hits to Github's API and even a new machine shouldn't be impacted?
slightly to attempt to update anyways
No worries, happy to test out as well if you want some test-cases.
while you wait for the next release you could delete ~/.cache/mise/ruby/ruby-build
before installing ruby which would prevent it from attempting to check the version
Okay thats not a bad workaround! Can you TL;DR me in that case how it understands what version to even install without hitting the Github API to check what to download?
lol: https://github.com/jdx/mise/blob/main/src/plugins/core/ruby.rs#L108-L109
lmk if you don't get it, it just git clone
's from master every time
Oh haha, yeah thats obvious now that I think about it 🙃
Describe the bug Looking at the documentation over here: https://mise.jdx.dev/configuration.html#mise-use-versions-host
Describes that this setting is default and should use the mise host instead of Github to fetch the current versions list and understand what to download.
Locally I am testing this and not seeing this behaviour, its constantly going to Githubs API to fetch the available versions. Reason this has come up is because we have been getting rate limited on the API and this seems to be the cause.
I looked into the code base to try and understand deeper and can only see this path being the only function that does not early return or use the
fetch_remote_versions_from_mise
function, so would be going to Github by default: https://github.com/jdx/mise/blob/f8ce139c1d0b41006dbbf1707801bf665f201ec6/src/plugins/core/ruby.rs#L355I am not a Rust developer so this is as far as my skillset will allow me to debug further unfortunately.
I even tried to override the environment variable
MISE_USE_VERSIONS_HOST=true
to see if the default wasn't working, but this had the same result.To Reproduce Install a version of ruby on your machine with
--verbose
mode and seemise
fetching the version from Github instead of themise
URL.Expected behavior Use the
mise
host instead of Github to prevent being rate limited.mise doctor
outputAdditional context This might be intended behaviour here in which case, any suggestions are welcomed!
I know I can fall back to using the
asdf
plugin which just has the build version hard coded and does agit
fetch instead of using the API at all, but would rather keep the core plugin here and see what can be done before doing that. Appreciate any help! :)