search

CH-Earth / datatool

datatool: Dynamical Data Processing Tool for HPCs
GNU General Public License v3.0
9 stars 4 forks source link

`canrcm4_wfdei_gem_capa` example not working #23

Closed ShervanGharari closed 1 year ago

ShervanGharari commented 1 year ago

The example provided for canrcm4_wfdei_gem_capa dataset hangs on log in node, while others, such as wrf-i, works fine. It would be great if you can take a look at this @kasra-keshavarz. Thank you,

ShervanGharari commented 1 year ago

Perhaps and as this dataset is not parallel, slurm may expect number of workers even if it is one after the update in july. Just guessing what might be the problem.

kasra-keshavarz commented 1 year ago

Good catch. I forgot to update the date chunking instructions for this dataset, when parallelizing feature based on ensemble members was added to the scripts. It should work fine now with 144b5f6bb803580d947e9d6f74f1e9b0d51d78c5.

This gives another parallelism scheme idea, based on variables, for datasets like this, where each file includes only 1 variable. I will add this to to-do list, and hopefully soon I can release another version.

Let me know if there is any other issues.