Possible improvements of simulation code

[tommy-engels]

This thread is a non-exhaustive list of ideas I have to improve the code performance.

1. Skip diagonal ghost node synching The diagonal ghost nodes are a significant overhead and I am unsure if they are actually required at all. for the differential operators no (only x,y,z = faces), and for CDF44 wavelets neither. Also, CDF40 does no longer check details on the ghost node layer, which was the case in old versions
2. Improve load balancing to take different computing time per block into account (from the mask generation, possibly chemistry as well)
3. Allow different N_ghostNodes for RHS and wavelet (because CDF44 requires g=6 but 4th order differences only g=3). This feature was enabled in the newGhostNodes branch.
4. openMP hybridization (take adavantage of odern, heterogeneous machines)

[tommy-engels]

An addition to 1): The problem will be the redundant nodes: if a block has a finer neighbor in the diagonal direction, then the common (redundant) corner point belongs to the coarse block (and not the fine one). It must be overwritten by the coarse value on the fine block. If ghost sync'ing is completely disabled in that direction, the redundant node is not overwritten (unless the other neighbor happens to be coarser as well, but that is not always the case)

This may lead to problems similar to what we had using "fineWins" for higher order lifted wavelets

[tommy-engels]

5. the code works using the refine-evolve-coarsening cycle, which is the important basis for our numerical method. Technically, it may be advantageous to modify this: coarsening can be done using a wavelet filter (and not removing the blocks), completely equivalent to the usual coarsening+refinement cycle. The refinement operator then only must refine blocks that have not been filtered. This may improve the adaptivity overhead, but possibly increases load balancing issues. It is thus not a priori clear if it will improve the simulation performance, but it is very likely.

[Arcadia197]

I would love to close this one as ideas are either longterm or finished / not possible, are you fine with it, @tommy-engels?