WW3 boundary condition issue

[yunfangsun]

Hi @aliabdolali @AliS-Noaa @sbanihash @pvelissariou1 ,

I was trying to follow https://polar.ncep.noaa.gov/waves/workshop/pdfs/WW3-workshop-exercises-day5-unstruct.pdf to update the boundary conditions for the Atlantic domain simulation.

The input files are directly downloaded from HPSS (/NCEPPROD/hpssprod/runhistory/rh${YY}/${YY}${MM}/${YY}${MM}${DD}/com_gfs_v16.2gfs.${YY}${MM}${DD}${i}.gfswave_output.tar), its format is like 'WAVEWATCH III SPECTRA' 50 36 1 'spectral resolution for points' 0.350E-01 0.375E-01 0.401E-01 0.429E-01 0.459E-01 0.491E-01 0.525E-01 0.562E-01 0.601E-01 0.643E-01 0.689E-01 0.737E-01 0.788E-01 0.843E-01 0.902E-01 0.966E-01 0.103E+00 0.111E+00 0.118E+00 0.127E+00 0.135E+00 0.145E+00 0.155E+00 0.166E+00 0.178E+00 0.190E+00 0.203E+00 0.217E+00 0.233E+00 0.249E+00 0.266E+00 0.285E+00 0.305E+00 0.326E+00 0.349E+00 0.374E+00 0.400E+00 0.428E+00 0.458E+00 0.490E+00 0.524E+00 0.561E+00 0.600E+00 0.642E+00 0.687E+00 0.735E+00 0.787E+00 0.842E+00 0.901E+00 0.964E+00 0.148E+01 0.131E+01 0.113E+01 0.960E+00 0.785E+00 0.611E+00 0.436E+00 0.262E+00 0.873E-01 0.620E+01 0.602E+01 0.585E+01 0.567E+01 0.550E+01 0.532E+01 0.515E+01 0.497E+01 0.480E+01 0.463E+01 0.445E+01 0.428E+01 0.410E+01 0.393E+01 0.375E+01 0.358E+01 0.340E+01 0.323E+01 0.305E+01 0.288E+01 0.271E+01 0.253E+01 0.236E+01 0.218E+01 0.201E+01 0.183E+01 0.166E+01 20220915 000000 '42003 ' 26.07 -85.94 3246.5 2.14 349.4 0.83 263.6 0.111E-17 0.207E-17 0.463E-17 0.226E-17 0.173E-17 0.754E-17 0.535E-17

and my ww3_bound.inp is as follow:

$ -------------------------------------------------------------------- $
$ WAVEWATCH III NetCDF boundary input processing                       $
$--------------------------------------------------------------------- $
$
$ Boundary option: READ or WRITE
$
 WRITE
$
$ Interpolation method: 1: nearest
$                       2: linear interpolation
 2
$ Verbose (0, 1, 2)
1
$
$ List of spectra files. These NetCDF files use the WAVEWATCH III
$ format as described in the ww3_ounp.inp file. The files are
$ defined relative to the directory in which the program is run.
$
$sci_ww3_spec_20220621.nc
station/gfswave.42058.spec
station/gfswave.42059.spec
station/gfswave.42060.spec
'STOPSTRING'
$
$ -------------------------------------------------------------------- $
$ End of input file                                                    $
$ -------------------------------------------------------------------- $

Whe I run ww3_bound > ww3_bound.out

It gave me the following errors:

  Grid name : atlantic

IP, file, I/O stat:    1, station/gfswave.42058.spec,     0
IP and spectral dimensions:    1   50   36
IP, file, I/O stat:    2, station/gfswave.42059.spec,     0
IP and spectral dimensions:    2   50   36
IP, file, I/O stat:    3, station/gfswave.42060.spec,     0
IP and spectral dimensions:    3   50   36

 *** WAVEWATCH III ERROR IN W3BOUND:
     ILLEGAL NK, NK = 32 DIFFERS FROM NKI = 50
     IT WILL BE MANAGED SOON BY SPCONV

The error is coming from ww3_bound.F90, 
      !
      ! Defines frequency range in spectra
      !

      ! Checks consistency of NK
      IF (NKI.GT.NK) GOTO 808
808 CONTINUE
  WRITE (NDSE,1008) NK, NKI
  CALL EXTCDE ( 68 )
  !
1008 FORMAT (/' *** WAVEWATCH III ERROR IN W3BOUND: '/               &
       '     ILLEGAL NK, NK =',I3,' DIFFERS FROM NKI =',I3/   &
       '     IT WILL BE MANAGED SOON BY SPCONV')

Could you please give me any suggestions on how to solve this error?

Thank you! my configuration is located at /scratch2/STI/coastal/Yunfang.Sun/ww3_hera/ian_obc on Hera

[aliabdolali]

the input BC has 50 freq, and 36 directional band. what is your simulation spectral resolution? They should match, otherwise you need to interpolate them yourself. My recommendation is to convert it to netcdf and I can give you scripts which does the conversion. Then you can use ww3_bounc instead of ww3_bound

[yunfangsun]

Hi @aliabdolali ,

I am using

SPECTRUM%NK        =  32
SPECTRUM%NTH       =  36

Would you happen to have any script to do the interpolation from 50 freq to 32 freq?

Thank you, could you please give me the script to convert the B.C. from text to NetCDF? I would like to try to use it.

Thank you again!

Best, Yunfang

[aliabdolali]

I have script which reads netcdf, does the interpolation and dump it in netcdf. You need to develop one to read ascii I assume as I do not have it. read netcdf: https://github.com/NOAA-EMC/WW3-tools/blob/develop/matlab_tools/swden_ww3_read.m an example of conversion and dump in netcdf: https://github.com/NOAA-EMC/WW3-tools/blob/develop/examples/Example_Directional_Spectral_Density_NDBC_nc.m just in https://github.com/NOAA-EMC/WW3-tools/blob/develop/examples/Example_Directional_Spectral_Density_NDBC_nc.m#L49, use swden_ww3_read

[aliabdolali]

Note that it is not only about matching no of freq and directions, but it should also match the frequency and directional bands.

[yunfangsun]

Hi @aliabdolali Thank you very much for your help! I will try to learn and use these functions.

[yunfangsun]

Hi @aliabdolali @sbanihash @AliS-Noaa ,

I could run the WW3 with open boundary conditions in my case, and there is no error message in the simulation, however, when I plot the results, and compare it with the simulation without obc, it shows the majority of the domain is NaN.

The run without obc shows a reasonable simulation:

However, when I add open boudaries, the results are like:

I have tried to use the original ASCII GFS wave model output, and the 50 spectral bin GFS netcdf file, and the interpolated 32 spectral bin netcdf file as the open boundries, all these cases have no error messages shown, but all the results are like above.

The log.ww3 shows:

WAVEWATCH III log file                                 version 7.14
 =====================================================================
                                                  date : 2024/03/20
                                                  time :  11:44:04

 Model definition file read.
 Restart file read; cold start (calm).

 Grid name : atlantic

 No water levels.
 No curents.
 Prescribed winds.
 No ice fields.
 No momentum
 No air density
 No mud density.
 No mud thickness.
 No mud viscosity.
 No ice parameter 1
 No ice parameter 2
 No ice parameter 3
 No ice parameter 4
 No ice parameter 5

 Gridded output fields :
--------------------------------------------------
     Water depth
     Wind speed
     Wave height
     Mean wave dir. a1b1

 Initial time     : 2022/09/23 00:00:00 UTC

                                       |         input         |      output      |
                                       |-----------------------|------------------|
     step | pass |    date      time   | b w l c t r i i1 i5 d | g p t r b f c r2 |
  --------|------|---------------------|-----------------------|------------------|
         0|     1| 2022/09/23 00:00:00 | F F                   | X                |
         1|     1|            00:01:40 | X                     |                  |
        36|     1|            01:00:00 |   X                   | X                |
        37|     2|            01:01:40 | X                     |                  |
        72|     2|            02:00:00 |   X                   | X                |
        73|     3|            02:01:40 | X                     |                  |
       108|     3|            03:00:00 |   X                   | X                |
       109|     4|            03:01:40 | X                     |                  |
       144|     4|            04:00:00 |   X                   | X                |
       145|     5|            04:01:40 | X                     |                  |

Could I know how to solve this issue? Thank you!

One of my test case is located at /scratch2/STI/coastal/Yunfang.Sun/ww3_hera/ian_obc_ascii_nc on Hera

[aliabdolali]

Hi @aliabdolali @sbanihash @AliS-Noaa ,

I could run the WW3 with open boundary conditions in my case, and there is no error message in the simulation, however, when I plot the results, and compare it with the simulation without obc, it shows the majority of the domain is NaN.

The run without obc shows a reasonable simulation:

However, when I add open boudaries, the results are like:

I have tried to use the original ASCII GFS wave model output, and the 50 spectral bin GFS netcdf file, and the interpolated 32 spectral bin netcdf file as the open boundries, all these cases have no error messages shown, but all the results are like above.

The log.ww3 shows:

WAVEWATCH III log file                                 version 7.14

 =====================================================================

                                                  date : 2024/03/20

                                                  time :  11:44:04

 Model definition file read.

 Restart file read; cold start (calm).

 Grid name : atlantic

 No water levels.

 No curents.

 Prescribed winds.

 No ice fields.

 No momentum

 No air density

 No mud density.

 No mud thickness.

 No mud viscosity.

 No ice parameter 1

 No ice parameter 2

 No ice parameter 3

 No ice parameter 4

 No ice parameter 5

 Gridded output fields :

--------------------------------------------------

     Water depth

     Wind speed

     Wave height

     Mean wave dir. a1b1

 Initial time     : 2022/09/23 00:00:00 UTC

                                       |         input         |      output      |

                                       |-----------------------|------------------|

     step | pass |    date      time   | b w l c t r i i1 i5 d | g p t r b f c r2 |

  --------|------|---------------------|-----------------------|------------------|

         0|     1| 2022/09/23 00:00:00 | F F                   | X                |

         1|     1|            00:01:40 | X                     |                  |

        36|     1|            01:00:00 |   X                   | X                |

        37|     2|            01:01:40 | X                     |                  |

        72|     2|            02:00:00 |   X                   | X                |

        73|     3|            02:01:40 | X                     |                  |

       108|     3|            03:00:00 |   X                   | X                |

       109|     4|            03:01:40 | X                     |                  |

       144|     4|            04:00:00 |   X                   | X                |

       145|     5|            04:01:40 | X                     |                  |

Could I know how to solve this issue? Thank you!

One of my test case is located at /scratch2/STI/coastal/Yunfang.Sun/ww3_hera/ian_obc_ascii_nc on Hera

Hi, Is field plot is the first time step or after a few,...? Your input might be contaminated by some nan values and they all spread all over the domain.

[yunfangsun]

Hi @aliabdolali ,

The problem is started from the first time step, and it is a cold-start case, and there is no problem for the cases without open boundary node conditions. And I have checked the wave boundary spectrum input, it looks like it doesn't contain NaN values.

[aliabdolali]

How many open boundary nodes do you have and how many spec.nc did you use? Share standard outputs of ww3_bounc with me

[yunfangsun]

Hi @aliabdolali ,

I have 334 open boundary nodes, and 106 spec.nc

  Summary grid statistics :
 --------------------------------------------------
       Number of longitudes      :         0
       Number of latitudes       :         0
       Number of grid points     :   2470094
       Number of sea points      :   2470094 (100%)
       Number of input b. points :       334
       Number of land points     :         0
       Number of excluded points :         0

And I have shared the open boundary related files (ww3_bounc.inp, ww3_bounc.out, nest.ww3) to google drive

[yunfangsun]

the link to the folder https://drive.google.com/drive/folders/1W43AWJUR7QEiPhm3UVjZXZdD_6L_QzHV?usp=sharing

[aliabdolali]

could you exclude the nodes that are too shallow from ww3_grid.nml and recreate mod_def.ww3 and try ww3_bounc and ww3_shel again? Also, remove spec.nc that are close to the shore. Also, I wonder if the depths in spec.nc files are correct? check them again. Note that the effect of OBC in shallow locations are negligible, so you can safely remove them.

[yunfangsun]

Hi @aliabdolali , Thank you for the suggestions, I will try it and let you know how it goes.

[yunfangsun]

Hi @aliabdolali ,

I have used (depth < -300m) in ww3_grid.inp, and (dpt > 300m) in the ww3_bounc.inp, where there are 225 obc nodes and 29 spectra inputs, and my working folder is /work2/noaa/nos-surge/yunfangs/ww3_hercules/ian_obc_ascii_nc_300m_04012024 on Hercules,

  Summary grid statistics :
 --------------------------------------------------
       Number of longitudes      :         0
       Number of latitudes       :         0
       Number of grid points     :   2470094
       Number of sea points      :   2470094 (100%)
       Number of input b. points :       225
       Number of land points     :         0
       Number of excluded points :         0

  End of program
 ========================================
         WAVEWATCH III Grid preprocessor

 SI nbPlus=    14713213  nbMinus=           5  nbZero=    14927910

However, from the first step the results for HS is still

Do you have any suggestions that I can try?

Thank you!

[aliabdolali]

I'd generate a synthetic spec and use it as boundary condition to isolate the issue, and see whether it is because of GFS or your configuration. Do you need a script to generate or you can take care of it yourself? Note that I do not work for NOAA and do not have access to NOAA HPCs.

[yunfangsun]

Hi @aliabdolali ,

It would be great if you could give me a script to generate a synthetic spec. Thank you!

[janahaddad]

@yunfangsun has WW3 with open BCs set-up working

[aliabdolali]

@yunfangsun has WW3 with open BCs set-up working

What was the issue?

[yunfangsun]

Hi @aliabdolali,

I removed some of the spectra forcing inputs (very near the shallow regions of the boundary), and the NaN values disappeared.

[sbanihash]

@yunfangsun did the run time improve as well?

[yunfangsun]

@sbanihash Yes, it only takes 1 hour for a 1-day simulation when I use 4500 processors for the mesh with 5 million elements on Hercules.

[aliabdolali]

Hi @aliabdolali,

I removed some of the spectra forcing inputs (very near the shallow regions of the boundary), and the NaN values disappeared.

Great, glad it worked. The problem is gfs data is taken from a coarse grid (13 km res) so near coast it is not representative. We do the same in nwps and other applications