search

softboiler / boilercv

Computer vision routines suitable for nucleate pool boiling bubble analysis
https://softboiler.org/boilercv/
MIT License
0 stars 2 forks source link

Investigate upstream issue with `netcdf4==1.7.1` on Linux. Pin `netcdf4` for now #213

Open blakeNaccarato opened 4 months ago

blakeNaccarato commented 4 months ago

Progress

Please check that this bug hasn't been reported before

Please confirm that this bug affects the latest version

Please indicate which operating system(s) you are using

Expected behavior

NetCDF files should be readable via xarray.open_dataset on all operating systems, including Linux.

Current behaviour

Receiving OSError: [Errno -101]. Some NetCDF files are failing to read in Linux CI. This also happens when tested locally in WSL2 Ubuntu. See the full traceback below.

Full traceback ```PowerShell Traceback (most recent call last): File "/home/user/code/mine/boilercv/.venv/lib/python3.11/site-packages/xarray/backends/file_manager.py", line 211, in _acquire_with_cache_info file = self._cache[self._key] ~~~~~~~~~~~^^^^^^^^^^^ File "/home/user/code/mine/boilercv/.venv/lib/python3.11/site-packages/xarray/backends/lru_cache.py", line 56, in __getitem__ value = self._cache[key] ~~~~~~~~~~~^^^^^ KeyError: [, ('/home/user/code/mine/boilercv/docs/data/uncompressed_filled/2023-09-20T17-14-18.nc',), 'r', (('clobber', True), ('diskless', False), ('format', 'NETCDF4'), ('persist', False)), '1a3953ac-3186-4b75-ad73-e4c909dfb9bd'] During handling of the above exception, another exception occurred: Traceback (most recent call last): File "", line 1, in File "/home/user/code/mine/boilercv/.venv/lib/python3.11/site-packages/xarray/backends/api.py", line 570, in open_dataset backend_ds = backend.open_dataset( ^^^^^^^^^^^^^^^^^^^^^ File "/home/user/code/mine/boilercv/.venv/lib/python3.11/site-packages/xarray/backends/netCDF4_.py", line 602, in open_dataset store = NetCDF4DataStore.open( ^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/code/mine/boilercv/.venv/lib/python3.11/site-packages/xarray/backends/netCDF4_.py", line 400, in open return cls(manager, group=group, mode=mode, lock=lock, autoclose=autoclose) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/code/mine/boilercv/.venv/lib/python3.11/site-packages/xarray/backends/netCDF4_.py", line 347, in __init__ self.format = self.ds.data_model ^^^^^^^ File "/home/user/code/mine/boilercv/.venv/lib/python3.11/site-packages/xarray/backends/netCDF4_.py", line 409, in ds return self._acquire() ^^^^^^^^^^^^^^^ File "/home/user/code/mine/boilercv/.venv/lib/python3.11/site-packages/xarray/backends/netCDF4_.py", line 403, in _acquire with self._manager.acquire_context(needs_lock) as root: File "/home/user/code/mine/boilercv/bin/python3.11/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/home/user/code/mine/boilercv/.venv/lib/python3.11/site-packages/xarray/backends/file_manager.py", line 199, in acquire_context file, cached = self._acquire_with_cache_info(needs_lock) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/code/mine/boilercv/.venv/lib/python3.11/site-packages/xarray/backends/file_manager.py", line 217, in _acquire_with_cache_info file = self._opener(*self._args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "src/netCDF4/_netCDF4.pyx", line 2470, in netCDF4._netCDF4.Dataset.__init__ File "src/netCDF4/_netCDF4.pyx", line 2107, in netCDF4._netCDF4._ensure_nc_success OSError: [Errno -101] NetCDF: HDF error: '/home/user/code/mine/boilercv/docs/data/uncompressed_filled/2023-09-20T17-14-18.nc' ```

Steps to reproduce

Install netcdf4==1.7.1 on Linux and run an affected notebook, e.g. docs/experiments/e230920_subcool/find_centers.ipynb.

Possible solution

Should visit upstream at xarray and netcdf4 to see what might be causing it.