Run May 2022 Event/debug fort.22* files

[ththelen]

@caseydietrich because I did not change the loop function in your NAM download code to loop over 30 days instead of 31, we now have this 9 digit jump in numbering moving from April 30 to May 1. However, we are not missing any on the NAM download files. Can I run the convert to fort.22 program with this jump in numbering, or do all the NAM download files need to be numbered perfectly sequentially (no jumps)?

[ththelen]

@caseydietrich I wrote a python script to correct this numbering jump so the question is now moot.

[ththelen]

@caseydietrich I am getting an error (snipped below) when I submit the grb2owi job to the queue. It looks like it has something to do with the wgrib2 executable. I moved this executable to the same folder where I was running the grb2owi script but that did not fix the problem. Can you provide any insight as to why I might be seeing this error?

[caseydietrich]

@ththelen In the queue submission script (grb2owi.csh), please add this line before the grb2owi script is called:

setenv PATH /full/path/to/grb2owi-master:$PATH

where you replace the full path to where that folder is located in your files. (cc: @anardek )

[ththelen]

Good morning @caseydietrich, the grb2owi script finished running yesterday. The script ran to completion and the only error message I saw is snipped below. Even with this 'FATAL ERROR' message, the script still produced a fort.221 and fort.222 file. I think the 043 NAM file was one of about 10 files I had to download by hand because the automatic download script timed out. It is strange that only this file out of the ones I manually downloaded and renamed prompted the error.

That said, the other 486 NAM files processed without incident, so I would like to believe the fort.22* files are fine even if one of the NAM timesnaps failed to process. Would you agree with this assessment?

[caseydietrich]

@ththelen The way to check is to type:

grep "DT=" fort.221 > junk

which will dump all of the snap header lines into a junk text file. In that file, you can check (a) does the file contain the correct overall number of snaps, and (b) do all of the snaps proceed in 3-hr increments. (cc: @anardek )

[ththelen]

@caseydietrich thanks for the tip. There was an erroneous time stamp. I redownloaded the 043 NAM file and re-ran the grb2owi program. The second time it ran without errors.

[ththelen]

River discharge for May event: 2000 ft^3/s = 60 m^3/s 60% of flow into estuary from Cape Fear River: 60 m^3/s / 0.6 = 100 m^3/s total flow rate to apply\ (note: Nov 2021 flow rate was 34 m^3/s)

specified flux per unit width (ADCIRC input for each river BC vertex) = (total flux) / (one half left element length + one half right element length) = (100 m^3/s) / (52 m) = 1.923 m^2/s flow rate for fort.15

[ththelen]

Water Level Offset Discussion item: what water level offset should we use for the May 2022 runs.

NOAA Wrightsville gauge: avg. verified - avg. predicted = 0.135 m NOAA Wilmington gauge: avg. verified -avg. predicted = 0.0565 m

[ththelen]

First run with old +0.15m offset crashed out. Using +0.1m for the offset as the average of Wilmington and Wrightsville

[ththelen]

@caseydietrich my Apr/May 2022 first leg run crashed out again. The error that I get in the padcswan run file is below.

... lots of iostat = 64

I check the headers for both the fort.221 and fort.222 files using your grep "DT=" fort.221 > junk command. Everything looks good there. Both files have the same number of lines (488), and have headers that proceed on in 3 hour increments like the following screen capture of the first several fort.221 file headers.

Do you have any suggestions for where I should start troubleshooting this error?

[caseydietrich]

@ththelen Can you please share your 221/222 files with me? Post them (in /share/jcdietri/For/Casey) and change the permissions (using 'chmod 777' and then the file/folder name). (cc: @anardek )

[ththelen]

@caseydietrich the fort.221 and 222 files can be found at the file path below:

/gpfs_common/share01/jcdietri/For/Casey/221111-May22Winds-Thelen

[caseydietrich]

@ththelen Thanks for sharing. These files contain only the header lines -- the fields of surface pressures and winds are missing. Were there any errors when you built the 221/222 files? (cc: @anardek )

[ththelen]

@caseydietrich the first time I ran the grb2owi script I got the error documented here: https://github.com/NCSU-CHAZ/Sunny-Day-Flooding/issues/20#issuecomment-1307213631

I deleted the fort.221/222 files from the previous grb2owi run, redownloaded the 043 NAM file, and re-ran grb2owi. The results are below. There are no error messages at any of the 488 time snaps.

time snaps 54-467. no error messages

[caseydietrich]

@ththelen I'm not sure what the error is. I will need to look at the wind files and script to try to debug, but they are too large to share. Can we look together on Mon morning? (cc: @anardek )

[ththelen]

@caseydietrich I am in Carolina Beach on Monday, but I am free Tuesday before 3:00 pm. If there is a time that works for you Tuesday before 3:00, let me know and I will make a calendar event.

When we meet, I would like to discuss the errors documented in this thread and the errors when I try to run the Nov 2021 simulation on the remeshed, cut down NC9 rev2.2 mesh. Those issues are (roughly) documented in the last couple posts in the thread linked here: https://github.com/NCSU-CHAZ/Sunny-Day-Flooding/issues/21#issuecomment-1312509499. I can provide more detail when we meet.

[caseydietrich]

@ththelen Sure thing, can you visit my office hours at Tue 1-2pm? (cc: @anardek )

[ththelen]

Yes, I'll be there

[ththelen]

@caseydietrich good news and bad news from our troubleshooting session this morning.

Bad news: fixing the environment path in the grb2owi.csh submission script did not fix the headers-only error in fort.22* files. Even with the corrected file path, we are not interpolating data from the NAM files, just reading in the headers. I will come to class this afternoon to discuss.

[ththelen]

@caseydietrich please find the .tgz file with all the NAML time snaps at the file path linked below. I think the permissions should be set such that you can view.

/gpfs_common/share01/jcdietri/For/Casey/221115-NAML-Thelen

[caseydietrich]

@ththelen I created the 221/222 files and posted them here:

/gpfs_common/share01/jcdietri/For/Thomas/221115-NAM

(cc: @anardek )

[ththelen]

@caseydietrich could you please confirm that the script to build the fort.22 files ran to completion? When I submitted the job this morning, the run errored out 2% of the way through the simulation. The error messages (copied below) look like the same messages I was seeing when I tried to submit the job with the headers only fort.22 files.

[ththelen]

@caseydietrich I scrolled through the whole fort.221 and fort.222 files in vi and I think I found the issue. It is not that the fort.22* builder did not run to completion. Instead, there is one time snap in the fort.221 file and three snaps in the fort.222 file that have headers only and no data associated with those 4 headers. The first of these is the 2022040115 time stamp called out in the error message above.

I think I can work around this problem by deleting the four headers that have headers only. Assuming there is nothing magical about the 3-hour time increment such that our simulation would crash if we skip a handful of time steps in the fort.22* files, I do not anticipate that missing this data will noticeably affect our runs.

Time steps missing fort data: fort.221: 2022040115 fort.222: 2022040715 202204110900 2022042421

[ththelen]

Update @caseydietrich: I just found this blurb in the ADCIRC fort.22 documentation. It looks like the 3 hour time step must be kept constant, so mix fix to delete headers missing data will not work. It looks like we will have to find a way to get the grb2owi script to run without outputting any blank headers

I am also concerned by the fact that ADCIRC was reading in data from April 1 at all. I am trying to run this simulation over the dates April 10-May 1. Does that mean my fort.221 and fort.222 files should start on April 10 as well instead of April 1 as we currently have it set up?

[caseydietrich]

@ththelen This is a good catch. I don't know why those four snaps would be skipped/omitted from the files. I tried to work-around this issue by copying the snaps immediately preceding those skipped snaps. For example, I copied the file for 2022040112 to also be used as 2022040115. This is sub-optimal, because the same wind snap would be used twice in a row, but it would keep the timing okay.

I did this and posted the files here:

/share/jcdietri/For/Thomas/221116-NAM

but they look to be the same size as the previous set, so I am not confident that this attempted work-around was successful. Do you have an automated way to check whether all snaps are present in the files?

To your other question, we need to adjust the fort.22 file to tell ADCIRC to skip the first snaps in the files. In the fort.22 file, the second line has an integer as the number of blank snaps to be inserted at the start of the simulation. If this number is positive, then ADCIRC will insert blank snaps. If this number is negative, then ADCIRC will skip that many snaps in the 221/222 files. So if you want to skip the first 14 days, you would use:

( 14 days ) ( 8 snaps per day ) ( -1 to tell ADCIRC to skip snaps ) = -112

(cc: @anardek )

[ththelen]

@caseydietrich I do not have an automated way to check this, but by opening the fort.22* in vi and searching for "20220" I can skip to each file header and look see if there is any data below it (e.g. below).

It looks like the new fort.22* files you shared are still missing data at the same snaps. I am going to try to copy the missing data from the prior time snaps using the vi text editor to implement your workaround. I just need to figure out the right vi keystrokes to pull this off

[caseydietrich]

@ththelen In that same folder, I added two more options: (1) repeating snaps as described above; (2) using only the snaps every 6 hr (instead of every 3 hr); and (3) redownloading (from the NOAA site) the four snaps that were troublesome. Please look at these files and see if any are promising. (cc: @anardek )

[ththelen]

@caseydietrich I went with option (1) and now have a run going. I also deleted out all snaps before April 10 since I was having problems getting the fort.22 set up correctly to start on the right snap. The run I have going now started on the correct OWI snap and has not thrown an error so far. Fingers crossed there are no typos in the fort.22* so the simulation runs to completion

[ththelen]

Good morning @caseydietrich, question about running the second leg of the May 2022 event here. Yesterday when we talked you said that the fort.26 file should have the start date of the second leg (May 9) as the start date as shown in Figure 1 below. My question now is about the fort.221 and fort.222 files. If I am using 0 as the NWBS value in my fort.22 file, should my fort.22* files to run the second leg have their first time snap as the start of the first leg (Apr 30) or as the start of the second leg (May 9)?

Figure 1

Figure 2

[ththelen]

Hello @caseydietrich and @anardek, I finished my write-up on the results for the May 2022 simulation. The attached Jupyter Notebook printout has water level, wind, and wave comparisons. Please do spend some time with this document when you have a chance. The document is fairly long because it contains all the code I used to import and plot data, but if you skip to headers/bolded sections and figures you should be able to navigate it fairly quickly.

CE596_Proj1_May2022_NC9rev2.1+0.1m.pdf