search

MeteoSwiss-APN / dawn

Compiler toolchain to enable generation of high-level DSLs for geophysical fluid dynamics models
MIT License
28 stars 30 forks source link

Change dawn-opt default behavior #1029

Open Stagno opened 3 years ago

Stagno commented 3 years ago

Using the new dusk cli (https://github.com/dawn-ico/dusk/pull/49) the following problem with dawn occurred to me. Behaviors of these two commands, which should do the same thing before code generation, differ:

dusk ./tests/examples/laplacian_fvm.py -generate-code -b ico-cuda -verbose
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Lowering.cpp:606] INFO: Intializing StencilInstantiation of `laplacian_fvm`
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Lowering.cpp:397] INFO: Processing vertical region at -1:-1
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Lowering.cpp:401] INFO: Inserting statements ... 
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Lowering.cpp:410] INFO: Inserted 7 statements
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Lowering.cpp:418] INFO: Filling accesses ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Lowering.cpp:643] INFO: Done initializing StencilInstantiation
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Lowering.cpp:669] INFO: Done initializing StencilInstantiation
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Driver.cpp:107] INFO: Starting parallelization passes for `laplacian_fvm` ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassInlining ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassInlining : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassFieldVersioning ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassFieldVersioning.cpp:154] INFO: laplacian_fvm: no rename
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassFieldVersioning : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassTemporaryType ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassTemporaryType.cpp:155] INFO: laplacian_fvm: demote: nabla2t2_vec
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassTemporaryType.cpp:155] INFO: laplacian_fvm: demote: nabla2t1_vec
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassTemporaryType : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassLocalVarType ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassLocalVarType : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassRemoveScalars ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassRemoveScalars : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassStageSplitAllStatements ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassStageSplitAllStatements : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassSetStageLocationType ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassSetStageLocationType : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassTemporaryType ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassTemporaryType.cpp:155] INFO: laplacian_fvm: promote: __local_nabl_71
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassTemporaryType.cpp:155] INFO: laplacian_fvm: promote: __local_nabl_70
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassTemporaryType : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassFixVersionedInputFields ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassFixVersionedInputFields : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassSetStageLocationType ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassSetStageLocationType : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassSetSyncStage ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassSetSyncStage : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassValidation ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassValidation : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Driver.cpp:112] INFO: Done with parallelization passes for `laplacian_fvm`
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Driver.cpp:178] WARNING: PassStageReordering currently disabled for unstructured meshes!
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Driver.cpp:243] WARNING: PassSetBlockSize currently disabled for unstructured meshes!
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Driver.cpp:287] INFO: Starting optimization and analysis passes for `laplacian_fvm` ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassSetStageName ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassSetStageName : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassValidation ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassValidation : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassSetStageGraph ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassSetStageGraph : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassSetDependencyGraph ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassSetDependencyGraph : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassStageMerger ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassStageMerger : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassTemporaryType ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassTemporaryType.cpp:155] INFO: laplacian_fvm: demote: __tmp_nab_71
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassTemporaryType.cpp:155] INFO: laplacian_fvm: demote: __tmp_nab_70
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassTemporaryType : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassLocalVarType ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassLocalVarType : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassRemoveScalars ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassRemoveScalars : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassSetSyncStage ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassSetSyncStage : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassValidation ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassValidation : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassSetCaches ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassSetCaches : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassValidation ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassValidation : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Driver.cpp:292] INFO: Done with optimization and analysis passes for `laplacian_fvm`
[/home/giacomo/dawn/dawn/src/dawn/CodeGen/Cuda-ico/CudaIcoCodeGen.cpp:1000] INFO: Starting code generation for ...
Assertion failed: `0' Var Access not allowed in this context
Function: 'virtual void dawn::codegen::cudaico::ASTStencilBody::visit(const std::shared_ptr<dawn::ast::VarAccessExpr>&)'
Location: /home/giacomo/dawn/dawn/src/dawn/CodeGen/Cuda-ico/ASTStencilBody.cpp:77
Aborted (core dumped)
dusk ./tests/examples/laplacian_fvm.py | dawn-opt --verbose
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Lowering.cpp:606] INFO: Intializing StencilInstantiation of `laplacian_fvm`
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Lowering.cpp:397] INFO: Processing vertical region at -1:-1
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Lowering.cpp:401] INFO: Inserting statements ... 
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Lowering.cpp:410] INFO: Inserted 7 statements
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Lowering.cpp:418] INFO: Filling accesses ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Lowering.cpp:643] INFO: Done initializing StencilInstantiation
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Lowering.cpp:669] INFO: Done initializing StencilInstantiation
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Driver.cpp:107] INFO: Starting parallelization passes for `laplacian_fvm` ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassInlining ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassInlining : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassFieldVersioning ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassFieldVersioning.cpp:154] INFO: laplacian_fvm: no rename
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassFieldVersioning : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassTemporaryType ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassTemporaryType.cpp:155] INFO: laplacian_fvm: demote: nabla2t2_vec
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassTemporaryType.cpp:155] INFO: laplacian_fvm: demote: nabla2t1_vec
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassTemporaryType : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassLocalVarType ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassLocalVarType : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassRemoveScalars ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassRemoveScalars : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassStageSplitAllStatements ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassStageSplitAllStatements : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassSetStageLocationType ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassSetStageLocationType : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassTemporaryType ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassTemporaryType.cpp:155] INFO: laplacian_fvm: promote: __local_nabl_131
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassTemporaryType.cpp:155] INFO: laplacian_fvm: promote: __local_nabl_130
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassTemporaryType : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassFixVersionedInputFields ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassFixVersionedInputFields : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassSetStageLocationType ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassSetStageLocationType : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassSetSyncStage ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassSetSyncStage : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassValidation ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassValidation : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Driver.cpp:112] INFO: Done with parallelization passes for `laplacian_fvm`
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Driver.cpp:287] INFO: Starting optimization and analysis passes for `laplacian_fvm` ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassInlining ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassInlining : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:46] INFO: Starting PassValidation ...
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/PassManager.cpp:59] INFO: Done with PassValidation : Success
[/home/giacomo/dawn/dawn/src/dawn/Optimizer/Driver.cpp:292] INFO: Done with optimization and analysis passes for `laplacian_fvm`

Specifically, it seems that the (default) passes run differ between the implementation of dawn-opt and dawn4py.compile. Compilation is failing in the dawn4py.compile path. This suggest that its set of default passes is wrong.

Stagno commented 3 years ago

Investigating this further i found out that the problem is that the dawn4py.compile path was running pass groups in the defaultPassGroups(): {PassGroup::SetStageName, PassGroup::StageReordering, PassGroup::StageMerger, PassGroup::SetCaches, PassGroup::SetBlockSize}, while the dawn-opt path was just lowering. StageMerger proved to be the problematic one because it generates local variables, not supported by cuda-ico (https://github.com/MeteoSwiss-APN/dawn/issues/1030).

Still it remains that the 2 paths differ substantially in which groups they run and, since one would not expect that to happen, we should uniform them.

Stagno commented 3 years ago

Decision taken was to also make dawn-opt run the defaultPassGroups() by default (without passing option --default-opt), to reflect the expectation that usually compilers apply some optimizations by default. Transforming this into a task.