Problem when running old benchmarks in Oceananigans

[francispoulin]

I am trying to run some benchmarks in oceananigans and when I do so it seems that a lot of packages need to be updated. You can see my current status below.

Ther is a lot of warnings but the error that made me post this is copied directly below.

Unfortunately, instantiate does not work. I don't use SparseArrays for anything else but this.

I am sorry that I don't have a lot of useful information but I'm happy to try anything that you might suggest. Just let me know.

ERROR: LoadError: ArgumentError: Package SparseArrays does not have SuiteSparse_jll in its dependencies:
- You may have a partially installed environment. Try `Pkg.instantiate()`
  to ensure all packages in the environment are installed.
- Or, if you have SparseArrays checked out for development and have
  added SuiteSparse_jll as a dependency but haven't updated your primary
  environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with SparseArrays

Current status of my julia library:

(Benchmarks) pkg> status
Project Benchmarks v0.1.0
Status `~/Software/Oceananigans.jl/benchmark/Project.toml`
⌃ [fbb218c0] BSON v0.3.5
⌃ [6e4b80f9] BenchmarkTools v1.3.1
⌅ [052768ef] CUDA v3.9.0
⌃ [a93c6f00] DataFrames v1.3.3
⌃ [7a1cc6ca] FFTW v1.4.6
⌃ [40713840] IncompleteLU v0.2.0
⌃ [42fd0dbc] IterativeSolvers v0.9.2
⌃ [033835bb] JLD2 v0.4.22
⌅ [da04e1cc] MPI v0.19.2
  [9e8cae18] Oceananigans v0.76.0 `..`
⌃ [bac558e1] OrderedCollections v1.4.1
  [32113eaa] PkgBenchmark v0.2.12
⌃ [91a5bcdd] Plots v1.28.1
⌅ [08abe8d2] PrettyTables v1.3.1
⌃ [d330b81b] PyPlot v2.10.0
⌅ [d496a93d] SeawaterPolynomials v0.2.2
  [56ddb016] Logging
Info Packages marked with ⌃ and ⌅ have new versions available. Those with ⌃ may be upgradable, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use `status --outdated`

[SobhanMP]

What version of Julia are you using?

[francispoulin]

Using 1.10.0

[ViralBShah]

Can you provide the exact way you check out Oceananigans, and the set of commands you run that produces that?

[francispoulin]

I decided to start from scratch as best i could

I first deleted my .julia file, hoping for a clean start. When I start julia there is nothing in my status. When I go into Oceananigans/benchmarks, I see there is stuff in my status, see below.

(Benchmarks) pkg> status
Project Benchmarks v0.1.0
Status `~/Software/TMP2_main/Oceananigans.jl/benchmark/Project.toml`
⌃ [fbb218c0] BSON v0.3.5
⌃ [6e4b80f9] BenchmarkTools v1.3.1
⌅ [052768ef] CUDA v3.9.0
⌃ [a93c6f00] DataFrames v1.3.3
⌃ [7a1cc6ca] FFTW v1.4.6
⌃ [40713840] IncompleteLU v0.2.0
⌃ [42fd0dbc] IterativeSolvers v0.9.2
⌃ [033835bb] JLD2 v0.4.22
⌅ [da04e1cc] MPI v0.19.2
  [9e8cae18] Oceananigans v0.76.0 `..`
⌃ [bac558e1] OrderedCollections v1.4.1
  [32113eaa] PkgBenchmark v0.2.12
⌃ [91a5bcdd] Plots v1.28.1
⌅ [08abe8d2] PrettyTables v1.3.1
⌃ [d330b81b] PyPlot v2.10.0
⌅ [d496a93d] SeawaterPolynomials v0.2.2
  [56ddb016] Logging

I start by doing instantaite but it seems to have some difficulties, as you can see below.

  151 dependencies successfully precompiled in 96 seconds. 8 already precompiled.
  2 dependencies had output during precompilation:
┌ LLVM
│  WARNING: could not import LLVMExtra_jll.libLLVMExtra into API
└  
┌ BFloat16s
│  WARNING: could not import Printf.ini_hex into BFloat16s
│  WARNING: could not import Printf.ini_HEX into BFloat16s
└  
  48 dependencies errored.
  For a report of the errors see `julia> err`. To retry use `pkg> precompile`

Maybe this is a problem with how Oceananigans has the benchmarks set up?

Anyhow, from Oceananigans/benchmarks I try running one of the examples and that's when I get an error about SuiteSparse_jll. See below for the beginning part.

julia> include("benchmark_shallow_water_model.jl")
Precompiling BenchmarkTools
        Info Given BenchmarkTools was explicitly requested, output will be shown live 
ERROR: LoadError: ArgumentError: Package SparseArrays does not have SuiteSparse_jll in its dependencies:
- You may have a partially installed environment. Try `Pkg.instantiate()`
  to ensure all packages in the environment are installed.
- Or, if you have SparseArrays checked out for development and have
  added SuiteSparse_jll as a dependency but haven't updated your primary
  environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with SparseArrays
Stacktrace:
  [1] macro expansion
...

Sorry for the mess but I hope this is reproducable by others.  I tried this on two different machines and found the same errors.

And I did try `resolve` and that didn't do much but showed the following information,

(Benchmarks) pkg> resolve ERROR: Unsatisfiable requirements detected for package CUDA [052768ef]: CUDA [052768ef] log: ├─possible versions are: 0.1.0-5.3.3 or uninstalled ├─restricted to versions 3.8.0-3 by Benchmarks [ee90295e], leaving only versions: 3.8.0-3.13.1 │ └─Benchmarks [ee90295e] log: │ ├─possible versions are: 0.1.0 or uninstalled │ └─Benchmarks [ee90295e] is fixed to version 0.1.0 └─restricted to versions 4.1.1-5 by Oceananigans [9e8cae18] — no versions left └─Oceananigans [9e8cae18] log: ├─possible versions are: 0.90.14 or uninstalled ├─restricted to versions * by Benchmarks [ee90295e], leaving only versions: 0.90.14 │ └─Benchmarks [ee90295e] log: see above └─Oceananigans [9e8cae18] is fixed to version 0.90.14

[fredrikekre]

Manifest files are tightly coupled with the Julia version that produced them. You have to either:

• Use the same Julia version that produced the manifest
• Tweak the CUDA compat here: https://github.com/CliMA/Oceananigans.jl/blob/d83a3f7aa21c463fc0cf2f930395cec05e014e74/benchmark/Project.toml#L25 to something that allows the same CUDA that Oceananigans itself uses and then resolve

[francispoulin]

Thanks @fredrikekre .

As pointed out here , I was in dependency hell. I removed some and added them back in and now evertyhing seems to be running normally.