mmbell / samurai

Spline Analysis at Mesoscale Utilizing Radar and Aircraft Instrumentation
GNU General Public License v3.0
13 stars 16 forks source link

Update the build and run scripts for NCAR machines #9

Closed sjsprecious closed 5 months ago

sjsprecious commented 5 months ago

This PR updates the build and run scripts for NCAR machines (Derecho and Casper). Specifically:

To switch the CPU/GPU configuration or change the XML file for a different case, revise ncar_build.sh and ncar_run.sh scripts accordingly.

The new script will also generate a run folder under the root directory and run a case there. After the simulation is done, it will generate another folder under the run folder, whose name is case-dependent, and move all the output to the newly generated folder. In this way, the root directory won't get messed up anymore.

mmbell commented 5 months ago

The capability to read and write from other directories is already present in the parameter specification. "data_directory" can provide an absolute path to the input location, and "output_directory" provides a path to the desired output location. I believe it will fail though if either of these two directories are missing rather than creating them. There is currently no capability to create a root/run directory that lives above the data/output, as this has typically been done manually by the user. If there is desire for a "run" directory from the script for use on NCAR machines that is fine, it just needs to then be consistent with data/output parameters or that run directory will probably be ignored.

sjsprecious commented 5 months ago

The capability to read and write from other directories is already present in the parameter specification. "data_directory" can provide an absolute path to the input location, and "output_directory" provides a path to the desired output location. I believe it will fail though if either of these two directories are missing rather than creating them. There is currently no capability to create a root/run directory that lives above the data/output, as this has typically been done manually by the user. If there is desire for a "run" directory from the script for use on NCAR machines that is fine, it just needs to then be consistent with data/output parameters or that run directory will probably be ignored.

Thanks Michael for providing this information. I did not realize those two options before.

We typically run SAMURAI through ./samurai /path_to_xml_file. Do you mean that we can also run something like ./samurai /path_to_xml_file --data_directory /path_to_data --output_directory /path_to_output?

In addition, the XML file already contains the path to the input data. Thus what is data_directory used for in this case?

sjsprecious commented 5 months ago

I still have concerns regarding the hardcoded paths to the executable used in the run scripts. We really need a more flexible approach that does not break if the scripts are used outside of the bin directory.

Thanks @johnmauff for your comment. I have updated the build and run scripts, and ask a user to explicitly set an environment variable named SAMURAI_ROOT so that we are able to run the NCAR scripts in a different path. I also moved all the NCAR scripts to a different folder named ncar_scripts.

johnmauff commented 5 months ago

Thank you for these modifications. These changes now look good. I am currently running tests on a single node of Derecho for the Beltrami, supercell, and hurricane test configurations.