Optimise gradient

[CyprienBosserelle]

Optimising gradient and gradient-halo

There is a new test to confirm the validity of the new gradient and gradient halo the speed gain. The new scheme is about 1.8x faster than original which should make the whole simulation 1.15x faster

This makes the model 22% faster than the current dev branch

• [x] Make New Halo script
• [x] Make new Gradient scheme
• [x] Write test for new halo scheme including speed test
• [x] write test for new gradient scheme including speed test
• [x] Fix any bugs in refine linear and conserve elevation that might cause instability at wet dry interface
• [x] add some documentation
• [x] double check gpu vs cpu
• [x] Add test to CI
• [x] Implement wet/dry instability fix for when conserve elevation is not requested
• [x] Test wet/dry instability implementation

[CyprienBosserelle]

There is a new test to confirm the validity of the new gradient and the speed gain. The new scheme is about 1.8x faster than original which should make the whole simulation 1.15x faster

[CyprienBosserelle]

Need to write a new test for the halo to verify where the bug is and once fixed whether this new scheme is worth it

[CyprienBosserelle]

The new halo scheme does not work and I'm not too sure why. a day of work didn't get me very far. I think I'm happy to drop that part and merge what I have done to move this along and possibly move on to more important problems.

[CyprienBosserelle]

Fixed another issue caused by my last commit. Although a bit unexpected.

Conserve elevation now works as expected.

Barycentric interpolation for the prolongation halo is necessary for checking whether neighbours are wet. Even if the halo value is re-calculated later when applying refine_linear.

[CyprienBosserelle]

There is another thing that could be done here but i'm not sure the consequence.

All of the instabilities (when not using Conserve Elevation) are because of the wet/dry front and varying resolution don't mix well. The solution to that issue in "Conserve elevation" uses a direct injection of values from the coarse wet cell for the halo rather than what come from refine linear. while "Conserv elevation" also does other things maybe we can use that prolongation even when not requesting conserve elevation. that would in effect only affect the cells at the wet/dry interface. The only issue would be that those cells do not conserve volume. does it mater so much?

• [x] Implement idea above
• [x] Test with ROG problem
• [x] Compare with Dev branch to see how bad it is.

[CyprienBosserelle]

Revert prolongation from conserve elevation when cells are fully wet?

Straight injection from Prolongation when neighbouring cell is dry affects the value of h, zs and zb. Also this action doesn't affect dzbdx. is that ok? The changes to zb should probably reverted at the end of the step. no?

On the other end, appling zb and h changes only to cells with wet neighbours sounds good but is a very inefficient action (50% slower!) so that should not be implemented.

• [x] Revert zb change at the end of the step for those cells affected by wet/dry

This seems to slow the code down a bit on the GPU (5-10%)

I wonder if we can reduce that so it does't try tp do anything on the GPU when blocks only have neighbour of the same level

[CyprienBosserelle]

should dhdx[halo] = T(0.0); in Conserve elevation?

Preformed a test to see the influence/impact of switching on/off the : dhdx[halo] = T(0.0); dzsdx[halo] = T(0.0);

in ProlongationElevationGH. This seem to have little effect.

[CyprienBosserelle]

Important note

Why is the calculation of dzbdx and dzbdy for GPU produced different result than CPU? The GPU side was replaced by a straight copy from CPU.

[CyprienBosserelle]

This merge request does a lot of different things to gradient, halo and Conserve elevation.

2 full scale test are needed to compare Dev branch and this branch

• [x] Rain on grid over Waikanae
• [x] Tsunami over Samoa

Once we are happy with the difference it will be ready to merge!

[CyprienBosserelle]

Waikanae Mass conservation test

Waikanae topo with cst rain at 50mm/h for 1 h. using the rainbnd option and wall bnd on all side. Results are undestinguishable from each other and after 1 hr the model has 100.48% of the theoretical volume of water in both with and without the wetdryfix.

For comparison the current Dev branch has 100.19% of the theoretical volume. While this is somehow better it does not really undermine the new branch (i.e. the dev branch might get closer to the theory for the wrong raisons), the bugfixes and instability improvement of this branch are totally justified.

results		% of theoretical volume	runtime
Wetdry fix	100.48%	97s
NO Wetdryfix	100.48%	96s
Development	100.18%	116s

[CyprienBosserelle]

Tsumani test results

The impact of the wet/dry fix should be much more obvious in tsunami simulation but here conserve elevation should be used in most tsunami cases.

No wetdryfix

switching off the wetdryfix causes instability that really have an impact on tsunami wave. Here the instability are clearly visible in the map and in the comparison are seen as 0.01 m waves before the tsunami wave arrive.

Wetdryfix

When using the wetdryfix the instabilities disappear and the solution is smoother.

Comparison

The instabilities are more obvious when we look at a transect diagonally from the plot above. These instabilities have a notable affect the height of the tsunami wave... a lot! This justifies that we should have the wetdryfix switched on by default.

Conserve Elevation

When running tsunami/storm surge simulation without rivers or rain then the conserve elevation should be switched on. This doesn't make a considerable impact on the tsunami wave (see transect above the red and blue line are indistinguishable) I'm not sure why the difference is so small but there is a difference. Below is the difference between the wetdryfix and conservelevation routine. it produces a difference of O(10-5).