search

NCAR / DART

Data Assimilation Research Testbed
https://dart.ucar.edu/
Apache License 2.0
184 stars 139 forks source link

Update WRF scripting and tutorial for derecho #636

Closed braczka closed 4 months ago

braczka commented 5 months ago

Description:

The csh scripting templates used to run WRF-DART and WRF-DART tutorial need to be updated from Cheyenne to Derecho. This PR includes functionality to get WRF functionality for version 3 (3.9.1), which was the version originally used when designing the tutorial. WRF Version 4 functionality will be included in a separate PR.

Fixes issue

This fixes issue #627.

Types of changes

Documentation changes needed?

Tests

I ran the existing WRF-DART Tutorial on Derecho and verified I could repeat the CONUS WRF test case and results. I compiled the WRFv3.9.1, WPSv3.9.1 and WRFDAv3.9.1 executables on Derecho using gfortran as covered in issue #627. Extending functionality beyond v3 to v4 will be included in a separate PR.

Checklist for merging

Checklist for release

Testing Datasets

The base directory for my WRF-Tutorial test case is located here: /glade/derecho/scratch/bmraczka/WRFv3.9.1_DART_Tutorial

braczka commented 5 months ago

Making this ready for review. Much of the scripting changes are self explanatory. For the mpi run command for either filter or the WRF executables I used the mpiexec command following Derecho documentation which specifies the number of processors and processors per node, but unsure I need to be this specific. CISL staff has resolved the module load difficulties I was having with the csh scripts (both command line execution and PBS submission) thus I did not need to include any special instructions there.

Because the observations were pre-prepared for the tutorial I did not troubleshoot or edit the observations section of the tutorial (Step 3). This does involve use of the prep_bufr obs converter, which I know there is active work being done on this #634. This may need to be incorporated in this pull request, not sure.

Within the documentation I try to give guidance on how to compile WRF and DART, even referring to a previous github issue where I go into details on environment, source code changes and configuration options. Not sure this is needed, given WRF is not our primary responsibility, but I think its helpful for DART users given the recent transition to Derecho. This PR provides a working version for WRFv3.9.1, and am still testing compatibility for WRVv4.x. This will be a separate PR.

mjs2369 commented 4 months ago

At the locations where we evaluate the logical if ( $SUPER_PLATFORM == 'yellowstone' ) then, should we use if ( $SUPER_PLATFORM == 'LSF system' ) then or something simalar instead since Yellowstone is decommissioned?

mjs2369 commented 4 months ago

Lastly, there are several issues related to "Step 3: Prepare observations [OPTIONAL]" part of the tutorial and the corresponding documentation page for PROGRAM prepbufr

These do not need to be addressed as a part of this PR.

Many of the prepqm files are “cosblocked”, which makes it so you can't untar them. On cheyenne there used to be a cosconvert program to fix the file, but this is not on Derecho. The majority of these prepqm files are therefore currently unusable. I followed the instructions here to build the program, but it was missing necessary files. Nancy was able to get the source code over email and build successfully. She is awaiting a response to her question of if we could put the source code some place users could access it. EDIT: we will not make the cosconvert source available or point to it - this is an RDA problem, not a DART problem

After discussion, we want to combine the information about prepbufr in this section of the WRF-DART tutorial with the info in the doc for the prepbufr program. It is difficult to follow the instructions and get all the necessary info having to go back and forth between these two documents.

In a separate pull request for the prepbufr documentation, we will remove the info in this section of the tutorial and instead just have it point to the prepbufr program page so we don't have to maintain two separate docs for this. It would also be good to include a note in the WRF-DART tutorial to use prepbufr obs from the year that matches the wrfinput files that come with the tutorial (2017)

We will also need to address several issues with the PROGRAM prepbufr page in addition to combining these two bits of documentation. See issue #634 for a list of these

braczka commented 4 months ago

Thanks @mjs2369 -- based on our last software standup discussion the prepbufr code discussion/resolution will continue on #634, but this PR can go forward as the WRF Tutorial documentation includes the converted obs already. I will add another another commit that includes some comments/warning within the observation section WRF documentation to keep users apprised of the prepbufr developments.

braczka commented 4 months ago

@mjs2369 Take one last look and then please merge into main at your discretion next week.

mjs2369 commented 4 months ago

At the locations where we evaluate the logical if ( $SUPER_PLATFORM == 'yellowstone' ) then, should we use if ( $SUPER_PLATFORM == 'LSF system' ) then or something simalar instead since Yellowstone is decommissioned?

@braczka what do you think about this?

braczka commented 4 months ago

At the locations where we evaluate the logical if ( $SUPER_PLATFORM == 'yellowstone' ) then, should we use if ( $SUPER_PLATFORM == 'LSF system' ) then or something simalar instead since Yellowstone is decommissioned?

@braczka what do you think about this?

Yes -- this seems reasonable to me, just make sure to do a search/replace for all instances of yellowstone in the code. I think I also mention Yellowstone instead of LSF system in the documentation as well.