Progress not displayed when using dynesty with MPI parallelization

Hi, I am using dynesty with MPI for parallel computation in my project, and I’ve encountered an issue with progress display. When running the code on a single core, the progress is displayed correctly as the computation proceeds. However, when running the same code using MPI for parallel processing, the progress is not shown during the calculation. Instead, the progress is only displayed either when the computation finishes or when it is interrupted midway. How can I display progress in real-time? Thanks in advance!

Environment: • dynesty version: 2.1.4 • mpi4py (Open MPI) version: 5.0.5 • Python version: 3.12.5 • OS: RockyLinux

Here is the relevant part of my code: (almost copy & paste from demo_mpi_params.py; the same issue occurs when running this demo file)

run_params["dynesty"] = True
run_params["emcee"] = False
run_params["optimize"] = False
run_params["nested_weight_kwargs"] = {'pfrac': 1.0}
run_params["nlive_init"] = 400
run_params["nlive_batch"] = 200
run_params["nested_dlogz_init"] = 0.1
run_params["nested_posterior_thresh"] = 0.1
run_params["nested_maxcall"] = int(1e7)
run_params["print_progress"] = True

mpi4py.rc.threads = False
mpi4py.rc.recv_mprobe = False

comm = MPI.COMM_WORLD
size = comm.Get_size()
rank = comm.Get_rank()

withmpi = size > 1

if withmpi:
        with MPIPool() as pool:
            if not pool.is_master():
                pool.wait()
                sys.exit(0)
            nprocs = pool.size
            print("Start fitting with MPI")
            output = fit_model(dict_obs, model, sps, pool=pool, queue_size=nprocs, lnprobfn=lnprobfn_fixed, **run_params)
    else:
        output = fit_model(dict_obs, model, sps, lnprobfn=lnprobfn_fixed, **run_params)

mpirun -np 8 python prospector_fit.py

bd-j / prospector

Progress not displayed when using dynesty with MPI parallelization #353