TRIQS / triqs

a Toolbox for Research on Interacting Quantum Systems
https://triqs.github.io
GNU General Public License v3.0
135 stars 71 forks source link

Broadcast of Python Gf fails for object sizes larger than 2GB #938

Open Wentzell opened 3 months ago

Wentzell commented 3 months ago

Description

See title

Steps to Reproduce

import numpy as np
from mpi4py import MPI

import triqs.utility.mpi as mpi
from [triqs.gf](http://triqs.gf/) import *

if mpi.is_master_node():
    g = Gf(mesh=MeshImTime(1., 'Fermion', n_tau=200), target_shape=[1000]*2)
    print(f'Size: {g.data.nbytes / 1024**3} GB')
    data = g.data
    data[:] = 1337
else:
    g = Gf(mesh=MeshImTime(1., 'Fermion', n_tau=200), target_shape=[1000]*2)
    data = g.data

# Breaks since size(g) = 3 GB, uses python pickle!
#g = mpi.bcast(g)

# Works since Bcast operates on existing buffers (no pickle involved).
data = MPI.COMM_WORLD.Bcast(data)

for rank in range(mpi.size):
    if mpi.rank == rank:
        print(rank)
        print(g)
        np.testing.assert_array_almost_equal(g.data, 1337 * np.ones_like(g.data))
    mpi.barrier()

Versions

Current unstable e1adae01c