Open johnomotani opened 6 months ago
@JoshuaLampert, can you take a look?
My current thought is that we probably should add HDF5_jll to the project if we are going to set a preference on it. The main question is can we do that for the user or not?
Before Julia 1.11 (master branch), adding a dependency on Pkg
was basically free, but from Julia 1.11, Pkg can be quite the dependency to load.
My main resolution here would be to add a friendlier message about adding HDF5_jll.jl to the project. Speculatively, we might be able to do the following for the user.
@eval Main begin
using Pkg
Pkg.add("HDF5_jll") # include UUID
end
If this (having to Pkg.add("HDF5_jll")
) is expected behavour, just having an error message saying this instead of the rather cryptic one in my original post would be good, and might be enough?
For me, it's not a problem to add HDF5_jll as long as I know that I'm supposed to have to do that, and it's not just a workaround for a bug.
IMO, it should also be possible without explicitly adding HDF5_jll. I look into it. But as a workaround you can add HDF5_jll to your project.
I cannot reproduce this. With an empty ~/.julia
directory in an empty project on julia 1.10.0:
julia> using Pkg
julia> Pkg.activate(".")
Activating new project at `~/test_hdf5_set_libraries`
julia> Pkg.status()
Status `~/test_hdf5_set_libraries/Project.toml` (empty project)
julia> Pkg.add("HDF5")
Installing known registries into `~/.julia`
[...]
julia> using HDF5
julia> hdf5_dir = "/usr/lib/x86_64-linux-gnu/hdf5/openmpi/"
"/usr/lib/x86_64-linux-gnu/hdf5/openmpi/"
julia> HDF5.API.set_libraries!(joinpath(hdf5_dir, "libhdf5.so"),
joinpath(hdf5_dir, "libhdf5_hl.so"))
[ Info: Please restart Julia and reload HDF5.jl for the library changes to take effect
EDIT: For me, HDF5_jll
is added to the [extras]
in the Project.toml when HDF5.API.set_libraries!
is called.
EDIT: As long as there are no preferences set, we load HDF5_jll and thus Preferences.jl finds it here. Once we have set the preferences, we have it in the Project.toml [extras]
and it is found here.
Did you have any LocalPreferences.toml in your active project before, @johnomotani? If yes, it should also work again after deleting the LocalPreferences.toml.
@JoshuaLampert thanks, yes it does work after deleting LocalPreferences.toml.
Sorry, I didn't work hard enough at making a reproducible example to start with! If I do what you did, but then delete everything in Project.toml while leaving LocalPreferences.toml there, then I get the error in the original post. Deleting LocalPreferences.toml fixes the problem. Doing Pkg.add("HDF5_jll")
also fixes the problem.
Yes, that makes sense. The problem only occurs when there are preferences in the LocalPreferences.toml and there is no HDF5_jll in the Project.toml. This is kind of an edge case since it only occurs when copying a LocalPreferences.toml file into a project or when manually deleting HDF5_jll from the Project.toml after setting the preferences, but not when setting the preferences normally. Should we catch this case anyway? If yes, I see two possibilities to deal with the situation:
Pkg.add
ing HDF5_jll.Pkg.add("HDF5_jll")
as suggested by @mkitti above, manually write HDF5_jll
to the [extras]
section of the Project.toml similar to what Preferences.jl does, or using HDF5_jll
(which also prevents Preferences.jl to raise the error). I'm not quite sure if the last two possibilities are feasible and don't have any negative side effects.What are your thoughts on this, @mkitti?
When I try to run
after starting with an empty
.julia
directory and with a new project, I often (always?) get an error like[I've edited out the full file paths as irrelevant]
If I explicitly install (i.e.
Pkg.add()
)HDF5_jll
as well asHDF5
then this error goes away, but I don't think I should have to do that, becauseHDF5_jll
is a dependency ofHDF5
. MaybeHDF5_jll
needs to be imported somewhere so thatset_libraries!()
can see it??Edit to add: This was using julia-1.10.0, HDF5.jl-0.17.1, and HDF5_jll.jl-1.12.2+2