search

COSIMA / cosima-cookbook

Framework for indexing and querying ocean-sea ice model output.
https://cosima-recipes.readthedocs.io/en/latest/
Apache License 2.0
58 stars 25 forks source link

Loading data from inaccessible location gives cryptic/unhelpful error #293

Open aidanheerdegen opened 2 years ago

aidanheerdegen commented 2 years ago

When cc.querying.getvar attempts to read data from an inaccessible location it gives a non-obvious error 

ValueError: did not find a match in any of xarray's currently installed IO backends ['netcdf4', 'h5netcdf', 'scipy', 'cfgrib', 'pydap', 'rasterio', 'zarr']. Consider explicitly selecting one of the installed engines via the ``engine`` parameter, or installing additional IO dependencies, see:
http://xarray.pydata.org/en/stable/getting-started-guide/installing.html
http://xarray.pydata.org/en/stable/user-guide/io.html

It is likely they have not joined the project necessary to access the data (cj50 and qv56 are the most obvious possibilities)

Click to show full error stacktrace ```python --------------------------------------------------------------------------- ValueError Traceback (most recent call last) /local/hh5/ed7737/tmp/ipykernel_2142946/2200008388.py in ----> 1 u = cc.querying.getvar(expt,'u', session) /g/data/hh5/public/apps/miniconda3/envs/analysis3-21.10/lib/python3.9/site-packages/cosima_cookbook/querying.py in getvar(expt, variable, session, ncfile, start_time, end_time, n, frequency, attrs, attrs_unique, **kwargs) 358 ncfiles = list(str(f.NCFile.ncfile_path) for f in ncfiles) 359 --> 360 ds = xr.open_mfdataset( 361 ncfiles, 362 parallel=True, /g/data/hh5/public/apps/miniconda3/envs/analysis3-21.10/lib/python3.9/site-packages/xarray/backends/api.py in open_mfdataset(paths, chunks, concat_dim, compat, preprocess, engine, data_vars, coords, combine, parallel, join, attrs_file, combine_attrs, **kwargs) 914 # calling compute here will return the datasets/file_objs lists, 915 # the underlying datasets will still be stored as dask arrays --> 916 datasets, closers = dask.compute(datasets, closers) 917 918 # Combine all datasets, closing them in case of a ValueError /g/data/hh5/public/apps/miniconda3/envs/analysis3-21.10/lib/python3.9/site-packages/dask/base.py in compute(traverse, optimize_graph, scheduler, get, *args, **kwargs) 569 postcomputes.append(x.__dask_postcompute__()) 570 --> 571 results = schedule(dsk, keys, **kwargs) 572 return repack([f(r, *a) for r, (f, a) in zip(results, postcomputes)]) 573 /g/data/hh5/public/apps/miniconda3/envs/analysis3-21.10/lib/python3.9/site-packages/distributed/client.py in get(self, dsk, keys, workers, allow_other_workers, resources, sync, asynchronous, direct, retries, priority, fifo_timeout, actors, **kwargs) 2689 should_rejoin = False 2690 try: -> 2691 results = self.gather(packed, asynchronous=asynchronous, direct=direct) 2692 finally: 2693 for f in futures.values(): /g/data/hh5/public/apps/miniconda3/envs/analysis3-21.10/lib/python3.9/site-packages/distributed/client.py in gather(self, futures, errors, direct, asynchronous) 1944 else: 1945 local_worker = None -> 1946 return self.sync( 1947 self._gather, 1948 futures, /g/data/hh5/public/apps/miniconda3/envs/analysis3-21.10/lib/python3.9/site-packages/distributed/utils.py in sync(self, func, asynchronous, callback_timeout, *args, **kwargs) 308 return future 309 else: --> 310 return sync( 311 self.loop, func, *args, callback_timeout=callback_timeout, **kwargs 312 ) /g/data/hh5/public/apps/miniconda3/envs/analysis3-21.10/lib/python3.9/site-packages/distributed/utils.py in sync(loop, func, callback_timeout, *args, **kwargs) 362 if error[0]: 363 typ, exc, tb = error[0] --> 364 raise exc.with_traceback(tb) 365 else: 366 return result[0] /g/data/hh5/public/apps/miniconda3/envs/analysis3-21.10/lib/python3.9/site-packages/distributed/utils.py in f() 347 if callback_timeout is not None: 348 future = asyncio.wait_for(future, callback_timeout) --> 349 result[0] = yield future 350 except Exception: 351 error[0] = sys.exc_info() /g/data/hh5/public/apps/miniconda3/envs/analysis3-21.10/lib/python3.9/site-packages/tornado/gen.py in run(self) 760 761 try: --> 762 value = future.result() 763 except Exception: 764 exc_info = sys.exc_info() /g/data/hh5/public/apps/miniconda3/envs/analysis3-21.10/lib/python3.9/site-packages/distributed/client.py in _gather(self, futures, errors, direct, local_worker) 1809 exc = CancelledError(key) 1810 else: -> 1811 raise exception.with_traceback(traceback) 1812 raise exc 1813 if errors == "skip": /g/data/hh5/public/apps/miniconda3/envs/analysis3-21.10/lib/python3.9/site-packages/dask/utils.py in apply() 35 def apply(func, args, kwargs=None): 36 if kwargs: ---> 37 return func(*args, **kwargs) 38 else: 39 return func(*args) /g/data/hh5/public/apps/miniconda3/envs/analysis3-21.10/lib/python3.9/site-packages/xarray/backends/api.py in open_dataset() 477 478 if engine is None: --> 479 engine = plugins.guess_engine(filename_or_obj) 480 481 backend = plugins.get_backend(engine) /g/data/hh5/public/apps/miniconda3/envs/analysis3-21.10/lib/python3.9/site-packages/xarray/backends/plugins.py in guess_engine() 153 ) 154 --> 155 raise ValueError(error_msg) 156 157 ValueError: did not find a match in any of xarray's currently installed IO backends ['netcdf4', 'h5netcdf', 'scipy', 'cfgrib', 'pydap', 'rasterio', 'zarr']. Consider explicitly selecting one of the installed engines via the ``engine`` parameter, or installing additional IO dependencies, see: http://xarray.pydata.org/en/stable/getting-started-guide/installing.html http://xarray.pydata.org/en/stable/user-guide/io.html ```

This is for documentation as much as anything (so people can search for their problem and find the solution). A nicer error message would be nice though.