Failed to precompile HTTP

[TonyGiorgio]

This is an existing instance that I'm trying to update.

Things I've tried:

git submodule update --recursive --remote
rm -rf ~/.julia
nix develop -c sh -c '$start_primal_caching_service'
rm -rf ~/.julia
nix --extra-experimental-features 'nix-command flakes' run github:NixOS/nixpkgs/23.11#nix -- develop -c sh -c '$start_primal_caching_service'

However, this works on a new directory on the same machine:

git clone --recurse-submodules https://github.com/PrimalHQ/primal-caching-service
cd primal-caching-service/
nix develop -c sh -c '$start_primal_caching_service'

ENV:

uname -a
Linux localhost 5.10.0-25-amd64 #1 SMP Debian 5.10.191-1 (2023-08-16) x86_64 GNU/Linux

  Downloaded artifact: OpenSSL
  Downloaded artifact: SQLite
  Downloaded artifact: Libiconv
  Downloaded artifact: OpenSpecFun
Precompiling project...
  ✗ HTTP
  83 dependencies successfully precompiled in 101 seconds
  1 dependency errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the package
Precompiling project...
  ✗ HTTP
  0 dependencies successfully precompiled in 2 seconds. 83 already precompiled.

ERROR: LoadError: The following 1 direct dependency failed to precompile:

HTTP [cd3eb016-35fb-5094-929b-558a96fad6f3]

Failed to precompile HTTP [cd3eb016-35fb-5094-929b-558a96fad6f3] to /home/primal/.julia/compiled/v1.8/HTTP/jl_ovAiyJ.
ERROR: LoadError: ArgumentError: Package HTTP does not have ExceptionUnwrapping in its dependencies:
- You may have a partially installed environment. Try `Pkg.instantiate()`
  to ensure all packages in the environment are installed.
- Or, if you have HTTP checked out for development and have
  added ExceptionUnwrapping as a dependency but haven't updated your primary
  environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with HTTP
Stacktrace:
 [1] macro expansion
   @ ./loading.jl:1167 [inlined]
 [2] macro expansion
   @ ./lock.jl:223 [inlined]
 [3] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1144
 [4] include(mod::Module, _path::String)
   @ Base ./Base.jl:419
 [5] include(x::String)
   @ HTTP /mnt/primal-db/primal-server/primal-caching-service/dev/HTTP.jl/src/HTTP.jl:1
 [6] top-level scope
   @ /mnt/primal-db/primal-server/primal-caching-service/dev/HTTP.jl/src/HTTP.jl:35
 [7] include
   @ ./Base.jl:419 [inlined]
 [8] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
   @ Base ./loading.jl:1554
 [9] top-level scope
   @ stdin:1
in expression starting at /mnt/primal-db/primal-server/primal-caching-service/dev/HTTP.jl/src/Exceptions.jl:1
in expression starting at /mnt/primal-db/primal-server/primal-caching-service/dev/HTTP.jl/src/HTTP.jl:1
in expression starting at stdin:1
Stacktrace:
 [1] pkgerror(msg::String)
   @ Pkg.Types /nix/store/i92h4wy9ay6iays1dsg99qsjsjybyli1-julia-bin-1.8.5/share/julia/stdlib/v1.8/Pkg/src/Types.jl:67
 [2] precompile(ctx::Pkg.Types.Context, pkgs::Vector{String}; internal_call::Bool, strict::Bool, warn_loaded::Bool, already_instantiated::Bool, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ Pkg.API /nix/store/i92h4wy9ay6iays1dsg99qsjsjybyli1-julia-bin-1.8.5/share/julia/stdlib/v1.8/Pkg/src/API.jl:1435
 [3] precompile
   @ /nix/store/i92h4wy9ay6iays1dsg99qsjsjybyli1-julia-bin-1.8.5/share/julia/stdlib/v1.8/Pkg/src/API.jl:1072 [inlined]
 [4] #precompile#225
   @ /nix/store/i92h4wy9ay6iays1dsg99qsjsjybyli1-julia-bin-1.8.5/share/julia/stdlib/v1.8/Pkg/src/API.jl:1071 [inlined]
 [5] precompile (repeats 2 times)
   @ /nix/store/i92h4wy9ay6iays1dsg99qsjsjybyli1-julia-bin-1.8.5/share/julia/stdlib/v1.8/Pkg/src/API.jl:1071 [inlined]
 [6] top-level scope
   @ /mnt/primal-db/primal-server/primal-caching-service/pkg.jl:1
in expression starting at /mnt/primal-db/primal-server/primal-caching-service/pkg.jl:1

[pritk]

in the old directory, is submodule dev/HTTP at commit a809094... ?

[pritk]

also, in the old directory, can you try

nix develop -c sh -c 'julia --project -e "import Pkg; Pkg.instantiate()"'

[TonyGiorgio]

I've figured out some of my problems. I've been running commands here instead of the repo I'm really using which is primal-server. For whatever reason I was also running into issues with the submodule not updating recursively into primal-caching-service which is what is pulling in HTTP.

Lastly, I'm having issues with conda not being installed. I see it was added last month, but has not been added to nix? I'll try pulling that into nix and seeing if that works.

https://github.com/PrimalHQ/primal-server/blame/8cbb7b11c87811f80425c5dd51c38d30142c4f64/src/media.jl#L425

[TonyGiorgio]

Tried a couple nix things but couldn't get it to work. I've just did a nix shell then entered julia and installed Conda that way. It's currently running and trying to process all of it's data, though I may have accidentally deleted a couple hundred GBs of notes but oh well. Will let you know if I run into any other snags, thank you.