Closed serbinsh closed 4 months ago
Looks like it's missing the h5 engine. I've run into this before and will try and update it to include the engine
@Ckster FYI - working on a notebook and ran into this issue again trying to use emit_tools. I think we need to be sure the HYR-SENSE kernal has the hdf5bindings installed so we can work through and develop our tutorials. Is it possible to make sure that gets added ASAP so we can work on these example?
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[7], line 1
----> 1 ds = et.emit_xarray(fp, ortho=True)
2 ds
File [~/HYR-SENSE/notebooks/agriculture/../../tools/emit/python/modules/emit_tools.py:61](https://a0db691fb.cyverse.run/lab/tree/HYR-SENSE/notebooks/agriculture/HYR-SENSE/tools/emit/python/modules/emit_tools.py#line=60), in emit_xarray(filepath, ortho, qmask, unpacked_bmask)
58 # Read in Data as Xarray Datasets
59 engine, wvl_group = "h5netcdf", None
---> 61 ds = xr.open_dataset(filepath, engine=engine)
62 loc = xr.open_dataset(filepath, engine=engine, group="location")
64 # Check if mineral dataset and read in groups (only ds[/loc](https://a0db691fb.cyverse.run/loc) for minunc)
File [/opt/conda/envs/hyr-sense/lib/python3.10/site-packages/xarray/backends/api.py:558](https://a0db691fb.cyverse.run/opt/conda/envs/hyr-sense/lib/python3.10/site-packages/xarray/backends/api.py#line=557), in open_dataset(filename_or_obj, engine, chunks, cache, decode_cf, mask_and_scale, decode_times, decode_timedelta, use_cftime, concat_characters, decode_coords, drop_variables, inline_array, chunked_array_type, from_array_kwargs, backend_kwargs, **kwargs)
555 if from_array_kwargs is None:
556 from_array_kwargs = {}
--> 558 backend = plugins.get_backend(engine)
560 decoders = _resolve_decoders_kwargs(
561 decode_cf,
562 open_backend_dataset_parameters=backend.open_dataset_parameters,
(...)
568 decode_coords=decode_coords,
569 )
571 overwrite_encoded_chunks = kwargs.pop("overwrite_encoded_chunks", None)
File [/opt/conda/envs/hyr-sense/lib/python3.10/site-packages/xarray/backends/plugins.py:205](https://a0db691fb.cyverse.run/opt/conda/envs/hyr-sense/lib/python3.10/site-packages/xarray/backends/plugins.py#line=204), in get_backend(engine)
203 engines = list_engines()
204 if engine not in engines:
--> 205 raise ValueError(
206 f"unrecognized engine {engine} must be one of: {list(engines)}"
207 )
208 backend = engines[engine]
209 elif isinstance(engine, type) and issubclass(engine, BackendEntrypoint):
ValueError: unrecognized engine h5netcdf must be one of: ['netcdf4', 'scipy', 'pydap', 'rasterio', 'store', 'zarr']
Thanks!
I was trying to experiment with this notebook to make sure we can manipulate EMIT data https://github.com/CU-ESIIL/HYR-SENSE/blob/serbinsh_ag/notebooks/agriculture/How_to_Convert_to_ENVI.ipynb
In our draft dependency check notebook I added listing the xarray backends and indeed hdf5 is missing
According to this the best way to install it into the image is
https://docs.xarray.dev/en/latest/getting-started-guide/faq.html
conda install -c conda-forge h5netcdf
But first in terminal had to do
jovyan@a0db691fb:~/HYR-SENSE$ conda info --env
# conda environments:
#
base /opt/conda
hyr-sense /opt/conda/envs/hyr-sense
jovyan@a0db691fb:~/HYR-SENSE$ conda activate hyr-sense
(hyr-sense) jovyan@a0db691fb:~/HYR-SENSE$
then install
I just manually installed into the kernel
conda install -c conda-forge h5netcdf
Re ran the depends check and hdf5 is there. going to test an example emit script now
Yup - after running that install the script is working
Thanks for finding the fix @serbinsh, h5netcdf was added to the environment.yml file and the new image was pushed to DockerHub. New deployments of the hyr-sense app will have it installed.
Thanks for finding the fix @serbinsh, h5netcdf was added to the environment.yml file and the new image was pushed to DockerHub. New deployments of the hyr-sense app will have it installed.
Excellent - thank you!
Using the HYR-SENSE kernel, hit a snag. Need to look into this to ensure we can open reflectance data following a similar notebook
My guess is its not loading
properly