search

geoschem / gcpy

Python toolkit for GEOS-Chem. Contains basic plotting scripts, plus the suite of GEOS-Chem benchmarking utilities.
https://gcpy.readthedocs.io
Other
51 stars 24 forks source link

[BUG/ISSUE] NotImplementedError when generating 1-year fullchem benchmark plots for GCHP #128

Closed msulprizio closed 3 years ago

msulprizio commented 3 years ago

When trying to generate the emission, J-value, and AOD plots for 1-year benchmark for 13.2.0, we get the following error:

%%% Creating GCHP vs. GCC emissions plots %%%

Creating plots for annual mean
Traceback (most recent call last):
  File "/n/holyscratch01/jacob_lab/msulprizio/GC/benchmarks/1yr/13.2.0-beta.1/GCHP/FullChem/BenchmarkResults/./run_1yr_fullchem_benchmark.py", line 991, in <module>
    bmk.make_benchmark_emis_plots(
  File "/n/home05/msulprizio/python/gcpy/gcpy/benchmark.py", line 1423, in make_benchmark_emis_plots
    results = Parallel(n_jobs=n_job)(delayed(createfile_hco_cat)(c)
  File "/n/seasasfs02/msulprizio/python/miniconda/envs/gcpy/lib/python3.9/site-packages/joblib/parallel.py", line 1041, in __call__
    if self.dispatch_one_batch(iterator):
  File "/n/seasasfs02/msulprizio/python/miniconda/envs/gcpy/lib/python3.9/site-packages/joblib/parallel.py", line 859, in dispatch_one_batch
    self._dispatch(tasks)
  File "/n/seasasfs02/msulprizio/python/miniconda/envs/gcpy/lib/python3.9/site-packages/joblib/parallel.py", line 777, in _dispatch
    job = self._backend.apply_async(batch, callback=cb)
  File "/n/seasasfs02/msulprizio/python/miniconda/envs/gcpy/lib/python3.9/site-packages/joblib/_parallel_backends.py", line 208, in apply_async
    result = ImmediateResult(func)
  File "/n/seasasfs02/msulprizio/python/miniconda/envs/gcpy/lib/python3.9/site-packages/joblib/_parallel_backends.py", line 572, in __init__
    self.results = batch()
  File "/n/seasasfs02/msulprizio/python/miniconda/envs/gcpy/lib/python3.9/site-packages/joblib/parallel.py", line 262, in __call__
    return [func(*args, **kwargs)
  File "/n/seasasfs02/msulprizio/python/miniconda/envs/gcpy/lib/python3.9/site-packages/joblib/parallel.py", line 262, in <listcomp>
    return [func(*args, **kwargs)
  File "/n/home05/msulprizio/python/gcpy/gcpy/benchmark.py", line 1397, in createfile_hco_cat
    compare_single_level(
  File "/n/home05/msulprizio/python/gcpy/gcpy/plot.py", line 858, in compare_single_level
    ds_dev_cmps[i] = regrid_comparison_data(
  File "/n/home05/msulprizio/python/gcpy/gcpy/regrid.py", line 634, in regrid_comparison_data
    new_data += regridder(data_reshaped[j])
  File "/n/seasasfs02/msulprizio/python/miniconda/envs/gcpy/lib/python3.9/site-packages/dask/array/core.py", line 1376, in __array_ufunc__
    return elemwise(numpy_ufunc, *inputs, **kwargs)
  File "/n/seasasfs02/msulprizio/python/miniconda/envs/gcpy/lib/python3.9/site-packages/dask/array/core.py", line 4365, in elemwise
    return handle_out(out, result)
  File "/n/seasasfs02/msulprizio/python/miniconda/envs/gcpy/lib/python3.9/site-packages/dask/array/core.py", line 4396, in handle_out
    raise NotImplementedError(msg)
NotImplementedError: The out parameter is not fully supported. Received type ndarray, expected Dask Array

The error was originally encountered using GCPy installed with conda-forge (Python 3.9.6), but Bob was also able to reproduce with his conda environment using Python 3.6.10 and older package versions.

msulprizio commented 3 years ago

The issue seems to be associated with the annual mean plots only. I'm able to generate the seasonal plots for Jan, Apr, July, and Oct without encountering this error.

lizziel commented 3 years ago

I'm going to take a look at this today and see if I can figure out what's up.

lizziel commented 3 years ago

I was able to fix this by removing one of the instances of the overloaded += operator within function regrid_comparison_data in regrid.py. The case it was being used for was not defined in the dask package. All other usages of += are simple integers so no problems there.

This fix is in commit https://github.com/geoschem/gcpy/commit/d851789644c3457166cffa674b5851d7d2035dae.