Initially the coupling blocks were constant and we could compute them on the global level. With the introduction of a sea ice mask there are now non-constant coupling coefficients in the coupling blocks. The quick and easy way to compute the state dependent coupling blocks is to use an AllGather() on the relevant state component (see e.g. https://github.com/nlesc-smcm/i-emic/blob/ea185fe1c7eff7acd509530826c01a73e6267f00/src/atmosphere/Atmosphere.C#L527). I expect this will affect the scaling behaviour.
erik808
commented
5 years ago
Initially the coupling blocks were constant and we could compute them on the global level. With the introduction of a sea ice mask there are now non-constant coupling coefficients in the coupling blocks. The quick and easy way to compute the state dependent coupling blocks is to use an
AllGather()on the relevant state component (see e.g. https://github.com/nlesc-smcm/i-emic/blob/ea185fe1c7eff7acd509530826c01a73e6267f00/src/atmosphere/Atmosphere.C#L527). I expect this will affect the scaling behaviour.It would be better to compute the coupling blocks locally. This would also benefit the restriction of physics to the local level. CouplingBlock::computeBlock() should be able to expect a local CRS struct. Currently only a warning is given: https://github.com/nlesc-smcm/i-emic/blob/ea185fe1c7eff7acd509530826c01a73e6267f00/src/coupledmodel/CouplingBlock.H#L175