Open bzah opened 2 years ago
Yes I think a dedicated chapter + a notebook is needed.
I have added a section in "the art of chunking" I think it make sense to have this within the dask/chunking/perfs chapter.
hi, happy to join the icclim community. I check the section on the art of chunking and gave it a try on TX90p. It seems I have an issue with chunking parameter which is not recognised:
CODE using icclim 5.1.0
with icclim.create_optimized_zarr_store(
in_files= ds,
var_names="tasmaxAdjust",
target_zarr_store_name="opti.zarr",
keep_target_store = False,
chunking={"time": -1, "i":"auto", "j":"auto" },
) as opti_tas :
icclim.index(
index_name="TX90p",
in_files=opti_tas,
slice_mode="month",
base_period_time_range=[base_dt1, base_dt2],
time_range=[dt1, dt2],
out_unit='%',
out_file=out_f_TX90p,
)
ERROR
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Input In [42], in <cell line: 6>()
1 start_time = time.time()
2 #catch wanring does not work like this
3 #with warnings.catch_warnings():
4 # warnings.filterwarnings(action='ignore', message='All-NaN slice encountered')
----> 6 with icclim.create_optimized_zarr_store(
7 in_files= ds,
8 var_names="tasmaxAdjust",
9 target_zarr_store_name="opti.zarr",
10 keep_target_store = False,
11 chunking={"time": -1, "i":"auto", "j":"auto" },
12 ) as opti_tas :
13 icclim.index(
14 index_name="TX90p",
15 in_files=opti_tas,
(...)
20 out_file=out_f_TX90p,
21 )
23 #ds_tx90p = icclim.index(index_name='TX90p', in_files=ds, var_name='tasmaxAdjust', slice_mode='month', base_period_time_range=[base_dt1, base_dt2], time_range=[dt1, dt2], out_unit='%', out_file=out_f_TX90p, logs_verbosity='HIGH')
File /usr/local/Cellar/python@3.9/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/contextlib.py:263, in contextmanager.<locals>.helper(*args, **kwds)
261 @wraps(func)
262 def helper(*args, **kwds):
--> 263 return _GeneratorContextManager(func, args, kwds)
File /usr/local/Cellar/python@3.9/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/contextlib.py:87, in _GeneratorContextManagerBase.__init__(self, func, args, kwds)
86 def __init__(self, func, args, kwds):
---> 87 self.gen = func(*args, **kwds)
88 self.func, self.args, self.kwds = func, args, kwds
89 # Issue 19330: ensure context manager instances have good docstrings
TypeError: create_optimized_zarr_store() got an unexpected keyword argument 'chunking'
Hi @scorbarieu, this is an issue with our documentation I was just fixing a few minutes ago ! The "chunking" attribute is only available in icclim 5.2 which is yet to be released.
You should check the stable version (5.1.0) of RTD here: https://icclim.readthedocs.io/en/stable/
In the future, "stable" documentation should be displayed by default on readthedocs and "latest" will be available on the version selector.
Note that, without "chunking" parameter, the create_optimized_zarr_store
behavior is to create a zarr store where time is unchunked and other dimensions are "auto" chunked.
Oh thanks sorry I did not pay attention! Thanks I will try that!
Description
The new rechunking feature is available but the documentation for it is limited.
We could either create a dedicated chapter (rst document) or add a section in our
dask.rst
.The doc should describe
Additionally, a didactic notebook using this feature would be great.