Slow `Pkg.update` on Windows

[sumiya11]

Hello! This looks and feels like a lot of time:

julia> @time begin import Pkg; Pkg.update("Groebner") end
    Updating registry at `C:\Users\User\.julia\registries\General`
    Updating git-repo `https://github.com/JuliaRegistries/General.git`
   Installed FastLapackInterface ─ v1.2.9
  No Changes to `C:\Users\User\.julia\environments\v1.8\Project.toml`
    Updating `C:\Users\User\.julia\environments\v1.8\Manifest.toml`
  [29a986be] ↑ FastLapackInterface v1.2.8 ⇒ v1.2.9
Precompiling project...
  7 dependencies successfully precompiled in 410 seconds. 414 already precompiled. 1 skipped during auto due to previous errors.
571.043752 seconds (9.39 M allocations: 775.004 MiB, 0.17% gc time, 0.34% compilation time)

Measured in a fresh Julia 1.8.2 session on a laptop running windows on intel i5 8250u. I do not know what is going on inside of Pkg.update, so I can not comprehend if this runtime is expected.

Still, I am wondering, where in precompilation does this runtime come from, and if I can do anything to improve it on my machine ?

julia> versioninfo()
Julia Version 1.8.2
Commit 36034abf26 (2022-09-29 15:21 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: 8 × Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, skylake)
  Threads: 1 on 8 virtual cores

[jebej]

It's the precompilation that's taking a long time, not the update part itself. You should be able to see what packages are being precompiled while the process is occuring, and so you can check which ones are taking so long. What packages are installed in your environment (Pkg.status())?

[sumiya11]

Thanks for your response!

In my first post, according to Pkg, it was precompiling 7 packages.

I posted the issue with this particular title after seeing that on Linux it can precompile 80 dependencies (with a similar environment and the same julia version) in under 250 seconds:

julia> @time Pkg.update()
...
Precompiling project...
  ✗ ExactSparseInterpolations
  ✗ GtkObservables
  ✗ ProfileView
  86 dependencies successfully precompiled in 239 seconds. 360 already precompiled.
  3 dependencies errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the packages
247.527285 seconds (6.98 M allocations: 586.983 MiB, 0.09% gc time, 0.15% compilation time)

Perhaps, I could split environment into smaller chunks to trigger less precompilation..

Environment on Windows:

julia> Pkg.status()
Status `C:\Users\User\.julia\environments\v1.8\Project.toml`
  [c3fe647b] AbstractAlgebra v0.27.10
  [1520ce14] AbstractTrees v0.4.4
⌅ [66b61cbe] AlgebraicSolving v0.2.2
  [6e4b80f9] BenchmarkTools v1.3.2
  [479239e8] Catalyst v13.0.0
  [861a8166] Combinatorics v1.0.2
  [adafc99b] CpuId v0.3.1
  [a93c6f00] DataFrames v1.5.0
  [864edb3b] DataStructures v0.18.13
  [0c46a032] DifferentialEquations v7.7.0
  [31c24e10] Distributions v0.25.81
  [fa6b7ba4] DualNumbers v0.6.8
  [7c1d4256] DynamicPolynomials v0.4.6
  [7da242da] Enzyme v0.10.18
  [790186a0] ExactODEReduction v0.1.0
  [4940a230] ExactSparseInterpolations v0.0.1 `https://github.com/sumiya11/ExactSparseInterpolations.jl#main`
⌅ [0b43b601] Groebner v0.2.11
⌅ [3e1990a7] Hecke v0.16.10
  [3e5b6fbb] HostCPUFeatures v0.1.14
  [7073ff75] IJulia v1.24.0
  [6deec6e2] IndexedTables v1.0.0
  [4138dd39] JLD v0.13.3
  [033835bb] JLD2 v0.4.30
  [682c06a0] JSON v0.21.3
  [51c06dcf] LispSyntax v0.2.1
  [bdcacae8] LoopVectorization v0.12.151
  [1914dd2f] MacroTools v0.5.10
  [961ee093] ModelingToolkit v8.47.0
  [102ac46a] MultivariatePolynomials v0.4.7
  [2edaba10] Nemo v0.32.7
  [e7bfaba1] NumericalIntegration v0.3.3
  [429524aa] Optim v1.7.4
⌃ [f1435218] Oscar v0.11.2
  [fae87a5f] ParserCombinator v2.1.1
  [2ae35dd2] Permutations v0.4.15
  [58dd65bb] Plotly v0.4.1
  [f0f68f2c] PlotlyJS v0.18.10
  [91a5bcdd] Plots v1.38.5
  [08abe8d2] PrettyTables v2.2.2
  [27ebfcd6] Primes v0.5.3
  [d330b81b] PyPlot v2.11.0
  [93e0c654] Reduce v1.2.12
  [ae029012] Requires v1.3.0
  [fdea26ae] SIMD v3.4.4
  [4edc584b] SIMDPolynomials v0.1.0
  [a2af1166] SortingAlgorithms v1.1.0
  [90137ffa] StaticArrays v1.5.16
  [f3b207a7] StatsPlots v0.15.4
  [220ca800] StructuralIdentifiability v0.4.7
  [a4af3ec5] SyntaxTree v1.0.1
  [98d24dd4] TestSetExtensions v2.0.0
  [3d5dd08c] VectorizationBase v0.21.59
⌃ [e88e6eb3] Zygote v0.6.44
  [56ddb016] Logging
  [9a3f8284] Random
  [8dfed614] Test

and on Linux:

julia> Pkg.status()
Status `~/.julia/environments/v1.8/Project.toml`
  [c3fe647b] AbstractAlgebra v0.27.10
⌅ [66b61cbe] AlgebraicSolving v0.2.2
  [6e4b80f9] BenchmarkTools v1.3.2
⌅ [2a0fbf3d] CPUSummary v0.1.30
⌃ [13f3f980] CairoMakie v0.6.3
⌃ [479239e8] Catalyst v12.3.2
  [861a8166] Combinatorics v1.0.2
  [adafc99b] CpuId v0.3.1
⌃ [a93c6f00] DataFrames v1.3.6
  [864edb3b] DataStructures v0.18.13
⌃ [0c46a032] DifferentialEquations v7.6.0
  [31c24e10] Distributions v0.25.81
  [7c1d4256] DynamicPolynomials v0.4.6
  [790186a0] ExactODEReduction v0.1.0 `https://github.com/x3042/Exact-reduction-of-ODE-systems#main`
  [4940a230] ExactSparseInterpolations v0.0.1 `https://github.com/sumiya11/ExactSparseInterpolations.jl#main`
⌅ [0b43b601] Groebner v0.2.11
  [3e5b6fbb] HostCPUFeatures v0.1.14
  [6deec6e2] IndexedTables v1.0.0
  [4138dd39] JLD v0.13.3
  [682c06a0] JSON v0.21.3
⌃ [bdcacae8] LoopVectorization v0.12.150
  [1914dd2f] MacroTools v0.5.10
  [102ac46a] MultivariatePolynomials v0.4.7
  [2edaba10] Nemo v0.32.7
⌃ [f1435218] Oscar v0.11.2
  [f0f68f2c] PlotlyJS v0.18.10
  [91a5bcdd] Plots v1.38.5
⌅ [d720cf60] Polymake v0.8.3
⌅ [08abe8d2] PrettyTables v1.3.1
  [27ebfcd6] Primes v0.5.3
  [c46f51b8] ProfileView v1.5.2
  [fdea26ae] SIMD v3.4.4
⌅ [bcd08a7b] Singular v0.14.0
  [f3b207a7] StatsPlots v0.15.4
  [123dc426] SymEngine v0.8.7
  [98d24dd4] TestSetExtensions v2.0.0
⌃ [3d5dd08c] VectorizationBase v0.21.58
  [de0858da] Printf
  [2f01184e] SparseArrays
  [8dfed614] Test

[ViralBShah]

It is better for this discussion to be on discourse.julialang.org. If it is determined to be an issue in Julia it would be best to reopen this issue with more detailed information.

[jebej]

FWIW, the package taking the most time is likely DifferentialEquations. But I agree, this conversation should be on Discourse.