search

cosmodesi / desilike

BSD 3-Clause "New" or "Revised" License
19 stars 14 forks source link

Differentiation logger printing duplicates with MPI #17

Open epaillas opened 1 year ago

epaillas commented 1 year ago

I'm running some BAO chains using emcee with multiple MPI processes. Some parameters are analytically marginalized over, and the logger is printing this line for every MPI process

[000008.12] [ 24/128] 08-18 10:55 Differentiation INFO Varied parameters: ['al0_-3', 'al0_-2', 'al0_-1', 'al0_0', 'al0_1', 'al2_-3', 'al2_-2', 'al2_-1', 'al2_0', 'al2_1']

I'm guessing the mesage is coming from https://github.com/cosmodesi/desilike/blob/e42e89c86a3667bb04a141be45e4e9d45047d747/desilike/differentiation.py#L234

which is supposed to only print it for the root process. Not sure why the logger is bypassing this conditional...

adematti commented 1 year ago

Hi, this is because when performing sampling, likelihood evaluations are distributed over MPI processes. Every process get to execute that differentiation part independently on a COMM_SELF (of size 1, so mpicomm.rank is always 0). Would you prefer this not to happen, or were you just wondering why this happened?