eWaterCycle / infra

Instructions for system administrators to deploy the eWaterCycle platform
Apache License 2.0
0 stars 0 forks source link

Wflow.jl fix: pin packages depending on netcdf/h5 binaries #153

Closed BSchilperoort closed 10 months ago

sverhoeven commented 10 months ago

I was able to reproduce error with:

rm -rf ~/.julia
cp -r /mnt/data/parameter-sets/wflow_humber ./
ipython
from pathlib import Path
from ewatercycle_wflowjl.model import WflowJl, install_
from ewatercycle.base.parameter_set import ParameterSet

parameters_humber = ParameterSet(
    name="humber",
    directory=Path("/home/sverhoeven/wflow_humber"),
    target_model="WflowJl",
    config="wflow_sbm.toml"   
)
model = WflowJl(
    parameter_set=parameters_humber,
)
cfg_file,_ = model.setup()
model.initialize(cfg_file)
[ Info: Initialize model variables for model type `sbm`.
[ Info: Cyclic parameters are provided by `/home/sverhoeven/wflow_humber/staticmaps.nc`.
[ Info: Forcing parameters are provided by `/home/sverhoeven/wflow_humber/inmaps-era5-2013_xaver.nc`.
/opt/conda/envs/ewatercycle/bin/python3.10: symbol lookup error: /home/sverhoeven/.julia/artifacts/87831472e1d79c45830c3d71850680eb745345fb/lib/libnetcdf.so: undefined symbol: H5Pset_all_coll_metadata_ops

Weirdly running without ewc/bmi works

from juliacall import Main
Main.Pkg.add(name="Wflow", rev="BMI")
Main.seval('using Wflow')
Main.seval('Wflow.run("wflow_sbm.toml")')
BSchilperoort commented 10 months ago

Weirdly running without ewc/bmi works

I suspect importing packages that depend on netcdf/hdf5 binaries adds these binaries PATH somehow. And then Julia uses the same environmental variables which are "polluted" by the ones from Python.

sverhoeven commented 10 months ago

No longer able to reproduce error on one of the src machines.