[NASA:Update] Distributed dace cache (rework)

[FlorianDeconinck]

Purpose

The original work to be able to compile DaCe orchestrated backend strategy was:

• Compile on 3x3 layout, compiling caches from .gt_cache_00000 -> .gt_cache_00008, reflecting the 9 possible codepath
• Any other layout would RUN mapping each rank to one of those 9 caches Unfortunately this is unreadable and demands a compile at 3x3 layout before doing any other runs

The new strategy cleans up code and actually generates the correct caches with any layout. E.g.

• 9 ranks will compile, tagging them TL, T, TR, R, BR, B, BL, L (for TopLeft, Top, etc.) which describes the actual code path on the cube-sphere face
• In RUN any ranks on any layout will compute which codepath they are part of and load accordingly

The same system should be deploy for gt backends, but is more complex due to the atomic nature of compiling, therefore is not part of this work.

This PR will synchronizes NASA & NOAA forks.

Code changes:

• ETA values for 137 levels
• GEOS wrapper bridge prints more informations

Checklist

Before submitting this PR, please make sure:

• [x] You have followed the coding standards guidelines established at Code Review Checklist.
• [x] Docstrings and type hints are added to new and updated routines, as appropriate
• [x] All relevant documentation has been updated or added (e.g. README, CONTRIBUTING docs)
• [x] For each public change and fix in pace-util, HISTORY has been updated
• [x] Unit tests are added or updated for non-stencil code changes

[FlorianDeconinck]

We are having a problem with determinism of temporaries. The utest fails sometimes, which sounds bad. But the regression test are passing fine.

[FlorianDeconinck]

We are having a problem with determinism of temporaries. The utest fails sometimes, which sounds bad. But the regression test are passing fine.

This is now solved.

[bensonr]

@FlorianDeconinck - in the orchestration.py source file, the comments highlighted are outdated with the new logic. Please update them.

[FlorianDeconinck]

All issues raised are logged.