search

JuliaLang / juliaup

Julia installer and version multiplexer
MIT License
1.02k stars 86 forks source link

`juliaup update` is initially slow to check versions db #1096

Open IanButterworth opened 2 weeks ago

IanButterworth commented 2 weeks ago

I've been looking into why juliaup update hangs for a while before doing any substantial downloads, even on very fast connections.

I think the issue is that this file may be served via a CDN that needs its caching settings tweaked to be less lazy https://julialang-s3.julialang.org/juliaup/RELEASECHANNELDBVERSION

It's a very small file, so doesn't make any sense that it takes 7s to download.. so I believe the time is the CDN refreshing a local cache.

From a profile during that quiet ~10s it can be seen that update_version_db is spending 7s waiting on reqwest.

Image

cc. @giordano @davidanthoff

davidanthoff commented 2 weeks ago

Yes, that makes a ton of sense: the idea is that this file is small so that it can be checked really quickly (clearly not working...). The file is also configured to not be cached by the CDN so that updates are picked up by clients quickly.

I'm not sure what the best strategy is, probably really depends on the CDN, right?

IanButterworth commented 2 weeks ago

Actually, I caught the slowdown with timing printing added and it seems to be something else.

This is with https://github.com/JuliaLang/juliaup/compare/main...ib/timing

Note I set the db patch version back one, just to exercise all code paths, but that is only hit after the problematic slow one.

% ./target/release/juliaup update
Checking for new Julia versions
Online db version `https://julialang-s3.julialang.org/juliaup/RELEASECHANNELDBVERSION`: Version { major: 1, minor: 0, patch: 55 }
Request to 'https://julialangnightlies-s3.julialang.org/bin/macos/aarch64/julia-latest-macos-aarch64.tar.gz' took 5.979535584s
Direct download etags: [("nightly", "\"2eeb5e6ccc5363b5180feb502f96d919-27\"")]
Bundled db version: Version { major: 1, minor: 0, patch: 54 }
Updating channel nightly

Then immediately after

ian@Ians-MacBook-Pro juliaup % ./target/release/juliaup update
Checking for new Julia versions
Online db version `https://julialang-s3.julialang.org/juliaup/RELEASECHANNELDBVERSION`: Version { major: 1, minor: 0, patch: 55 }
Request to 'https://julialangnightlies-s3.julialang.org/bin/macos/aarch64/julia-latest-macos-aarch64.tar.gz' took 112.53425ms
Direct download etags: [("nightly", "\"2eeb5e6ccc5363b5180feb502f96d919-27\"")]
Bundled db version: Version { major: 1, minor: 0, patch: 54 }