Open holke opened 5 months ago
@Davknapp @lukasdreyer ping
@Davknapp: I think there is a similar problem, for t8_forest_partition_create_tree_offsets
after calling t8_forest_partition_given
, when you have 2 empty procs next to each other.
In t8_forest_partition_given
, the following lines set first and last local tree in a way, that does not fit the offset rule needed to compute num_local_trees correctly at a different place. Therefore, offset is computed to be 0, and not the treeid of the next tree.
This bug can be seen by executing
mpirun -n 8 ./test/t8_forest/t8_gtest_forest_commit --gtest_filter=*prism_cake*
When constructing a cmesh. If t8_cmesh_load is used and the current process is empty, then the cmesh sets its num_local_tree to 0, its first_tree to -1 and first_tree_shared to 0.
This can break the offset computation for empty processes in
t8_cmesh_gather_treecount_ext
, becauset8_offset_next_nonempty_rank
may identify an empty rank as nonempty in the case that multiple empty are in a row.Note, that this behaviour may also happen in potential new cmesh construction methods (that do not exist yet, but use a similar logic to set the empty processes).
Here is how we can fix the behaviour:
t8_cmesh_gather_treecount_ext
. Instead use a second array of bool/int with values 0 and 1 for empty or non-empty processes and Allgather that array as well.t8_offset_next_nonempty_rank
t8_cmesh_offset_empty_correction