Generalised topology construction (#2306) yields invalid partitioning for simple mesh

[anzil]

For a simple mesh (4 nodes, 3 cells, gdim=3, tdim=1) the partitioning using #2306 results in

• an invalid global size of the index map for selected topological dimensions,
• incorrect global indices in the index map for selected topological dimensions

as demonstrated in the minimal example below.

import dolfinx
import ufl
import numpy as np
from mpi4py import MPI

comm = MPI.COMM_WORLD

#         (3)
#          |
#          |[2]
#          |
# (1)-----(0)-----(2)
#     [0]     [1]

if comm.rank == 0:
    coord = np.array([[0.0, 0.0, 0.0], [-1.0, 0.0, 0.0], [+1.0, 0.0, 0.0], [0.0, +1.0, 0.0]])
    cells = np.array([[0, 1], [0, 2], [0, 3]])
else:
    coord = np.ndarray((0, 3))
    cells = np.ndarray((0, 2))

ufl_cell = ufl.Cell("interval", geometric_dimension=3)
ufl_elem = ufl.FiniteElement("Lagrange", ufl_cell, 1)
ufl_mesh = ufl.Mesh(ufl.VectorElement(ufl_elem))

mesh = dolfinx.mesh.create_mesh(comm, cells, coord, ufl_mesh)

for tdim in range(mesh.topology.dim + 1):
    imap = mesh.topology.index_map(tdim)
    lidx = np.arange(imap.size_local, dtype=np.int32)
    size = imap.size_global
    gidx = imap.local_to_global(lidx)
    print(f"(tdim={tdim}, rank={comm.rank}): size_global = {size}, global_indices = {gidx}")

Before the changes in #2306:

dolfinx@136aa0acdeeb7e29b336949ad9ed8537237f5f88# mpirun -n 2 python3 mfe.py 
(tdim=0, rank=0): size_global = 4, global_indices = [0]
(tdim=1, rank=0): size_global = 3, global_indices = [0]
(tdim=0, rank=1): size_global = 4, global_indices = [1 2 3]
(tdim=1, rank=1): size_global = 3, global_indices = [1 2]

After the changes in #2306:

dolfinx@dfbcb44ad7e109a3367f8e7c0f973487f58eafbe# mpirun -n 2 python3 mfe.py 
(tdim=0, rank=0): size_global = 5, global_indices = [0 1]
(tdim=1, rank=0): size_global = 3, global_indices = [0]
(tdim=0, rank=1): size_global = 5, global_indices = [2 3 4]
(tdim=1, rank=1): size_global = 3, global_indices = [1 2]

For tdim=0 (nodes) size_global is wrong (5!=4) and global indices contain invalid entries.

[garth-wells]

@chrisrichardson do you have a fix for this?

[jorgensd]

A suggested fix for this issue has been outlined in: https://github.com/FEniCS/dolfinx/issues/3063#issuecomment-1962057089