search

NCAR / CUPiD

CUPiD is a “one stop shop” that enables and integrates timeseries file generation, data standardization, diagnostics, and metrics from all CESM components.
https://ncar.github.io/CUPiD/
Apache License 2.0
21 stars 19 forks source link

PR for Issue #89: "Ensure CUPiD is generalizable and portable to other machines" #92

Open shivaniikum opened 2 months ago

shivaniikum commented 2 months ago

This pull request addresses issue #89 and contains changes to the config_f.cam6_3_119.FLTHIST_ne30.r328_gamma0.33_soae.001.yaml file at path "examples/nblibrary/config_f.cam6_3_119.FLTHIST_ne30.r328_gamma0.33_soae.001.yaml".

I've removed hardcoded references to glade/ and added global variables to the top of the file and changed to relative file paths throughout the rest of the file. (Note: users should change the start and end year variables to match their cesm output files).

TeaganKing commented 2 months ago

Hi @shivaniikum , thanks for getting this PR started, and nice work finding all of these glade paths.

One main thing that I'm noticing is that we want to generalize how we read in files while also maintaining the functionality of pointing to the current paths.

So, I think it would be useful to set a root directory in a config.yml file parallel to the examples/coupled_model directory (rather than the one in nblibrary), and use this to determine the full paths in config_f.cam6_3_119.FLTHIST_ne30.r328_gamma0.33_soae.001.yaml For instance, the root_directory variable could be something like ../local_cesm_output/ in your config.yml file, and something like /glade/campaign/cesm/development/cross-wg/diagnostic_framework/ in the examples/coupled_model config file.

@mnlevy1981 do you want to add anything else to this suggestion?

mnlevy1981 commented 2 months ago

Nothing to add to your specific suggestion, but I also wanted to ask @shivaniikum to merge the latest main into this branch since it was started before #91 was merged it

mnlevy1981 commented 2 months ago

Actually, one thing to add - we're okay with modifying the directory structure under /glade/campaign/cesm/development/cross-wg/diagnostic_framework/, so if setting a root directory in examples/coupled_model/config.yml results in slightly different paths being passed to ADF that's fine. At some point in the future we'll decide on a file structure assuming the root directory is a path generated by CESM, and then we'll make sure any existing examples fit, but until then the directory structure we talked about last week is preferable to what currently exists on the NCAR machine.

shivaniikum commented 2 months ago

Thank you for reviewing this PR @mnlevy1981 and @TeaganKing. I will work on making these changes.

mnlevy1981 commented 2 months ago

Two more requests before digging in for an actual review:

  1. It looks like your config.yml file that gets data from examples/local_cesm_output/ hasn't been added to this PR yet (if the decision is to make it examples/local_cesm_output/config.yml, then your root directory could be ./ instead of ../local_cesm_output)
  2. Could you please provide a small README in addition to config.yml describing how to get the necessary data from the web?