search

wrf-model / WRF

The official repository for the Weather Research and Forecasting (WRF) model
Other
1.25k stars 683 forks source link

Halo issue with feedback for large odd nest ratios (> 5:1) #1539

Open rsarthur opened 3 years ago

rsarthur commented 3 years ago

Describe the bug When feedback is used with a nest ratio larger than 5:1, halo errors appear within the nest region of the parent domain.

To Reproduce Steps to reproduce the behavior:

  1. Version 4.3 with configure option 15 (INTEL ifort/icc, dmpar)
  2. Pertinent namelist options: feedback=1, parent_grid_ratio>=5 (see attached namelists) for tests below)
  3. See below for description of tests/output

Expected behavior When feedback is on, we expect "smooth" variable fields in the nest region of the parent domain. However, errors appear when the model is run in parallel, seemingly due to a halo issue.

Screenshots Test results (see description below) for various configurations:

WRF v4.3 with varying nest ratios: vary_nr

Nest ratio of 9 with various code versions and parallel settings: vary_code

Attachments input_sounding.txt namelist.input.nest_ratio_3.txt namelist.input.nest_ratio_5.txt namelist.input.nest_ratio_7.txt namelist.input.nest_ratio_9.txt

Additional context The results shown above are for an idealized, 2-domain nested setup with feedback turned on. Screenshots show MU on d01 after 1 second of run time. Note that the nest regions do not exactly align as the nest ratio is varied. We believe the small variations seen for nest ratios of 3 and 5 (and other well behaved cases, ie the plots that are mostly pink) are acceptable "base level" variations due to feedback - they are much smaller than the suspected halo errors for nest ratios of 7 and 9. Although only MU is shown, similar errors appear in other variables as well.

This issue appears to be related to PR #1100. When the test case is run in a version prior to this PR (we used version 4.0.1) or in version 4.3 with the code changes from this PR reverted, there is no halo issue. Furthermore, there are no obvious halo issues on the nest domain, so the error seems to result from the feedback step on d01. I am hopeful that updating a halo exchange somewhere will fix this issue, but I have not been able to fix it myself.

davegill commented 3 years ago

@rsarthur Robert, This is about the nicest issue I have seen anyone put together.

rsarthur commented 3 years ago

Ha! Thank you, I'm flattered. I hope we can fix it, please let me know if I can provide any other details.