search

NCAR / CUPiD

CUPiD is a “one stop shop” that enables and integrates timeseries file generation, data standardization, diagnostics, and metrics from all CESM components.
https://ncar.github.io/CUPiD/
Apache License 2.0
21 stars 19 forks source link

Pass paths via config #55

Closed mnlevy1981 closed 5 months ago

mnlevy1981 commented 5 months ago

Use config.yml to pass configuration details (such as the directory containing CESM output) to both land_comparison.ipynb and ocean_surface.ipynb. Note that both of those are new names for old notebooks -- the land notebook had a typo in the name (comparison was misspelled) and the ocean notebook had the more generic name surface.ipynb

mnlevy1981 commented 5 months ago

I removed the dask client from the ocean notebook, and the land & atmosphere notebooks were both already serial. On derecho, it took ~11 minutes to run:

(cupid-dev) $ time cupid-run config.yml
real    11m2.020s
user    14m50.714s
sys 0m28.384s
(cupid-dev) $
mnlevy1981 commented 5 months ago

729064f switches to the serial task manager, which lets users follow progress on stdout. On derecho, it took a little longer than the parallel manager (11:30 instead of 11:02), which I think is a worthwhile trade-off:

real    11m35.800s
user    4m16.428s
sys 0m22.316s
TeaganKing commented 5 months ago

I just made a few comments that could be addressed either in this PR or at a later time. Overall this looks good to me, though!