emscripten-forge / recipes

Recipes to build the packages for the emscripten-forge distribution
BSD 3-Clause "New" or "Revised" License
56 stars 49 forks source link

Add h5py #1354

Open talmo opened 1 month ago

talmo commented 1 month ago
talmo commented 1 month ago

At this point I think I'm stuck. The package builds, but then we get loads of errors deep in HDF5. There are plenty of suspicious warnings earlier on, but unclear where it's coming from.

Maybe @DerThorsten or @bmaranville have some ideas?

talmo commented 1 month ago

Perhaps the hdf5 package in this repo needs to be built as a shared library like libhdf5 over in Pyodide?

DerThorsten commented 1 month ago

At this point I think I'm stuck. The package builds, but then we get loads of errors deep in HDF5. There are plenty of suspicious warnings earlier on, but unclear where it's coming from.

Maybe @DerThorsten or @bmaranville have some ideas?

I might find some time to look into this, but I would also need to do some major debugging / experimentation

bmaranville commented 1 month ago

Sorry, I can't be much help until I understand how this very specific build process works...

I went through the instructions for doing a local build, but after I build libhdf5 and then h5py, I get an error:

rattler-build build  --recipe recipes/recipes_emscripten/h5py/recipe.yaml  --target-platform=emscripten-wasm32 -c https://repo.mamba.pm/emscripten-forge -c conda-forge -c microsoft -m conda_build_config.yaml
...
Error building package: Failed to resolve dependencies: Cannot solve the request because of: No candidates were found for libhdf5 *

How do you tell the h5py build to use the locally-built libhdf5?

talmo commented 1 month ago

I think the issue is that we're building two conda packages successfully (libhdf5 and h5py), but we're not telling the second one (h5py) where to find its dependency (libhdf5).

With conda-build I think you have to set the local build directory as an additional channel to look for locally-built packages, but I'm not so sure how to do that in rattler-build.

Alternatively, we could get libhdf5 merged in and uploaded to the repository so that h5py can find it, but it'd be nice to be able to test it first before uploading.

Maybe @DerThorsten or @wolfv can provide some quick assistance? I think the hard part is done (the packages individually are getting built!).

DerThorsten commented 1 month ago

@talmo I hope I will find some time to debug this, but cannot promise to do It this week!