Trouble with alive_bar in a mpi application

[alentner]

I really enjoy using alive_progress in my projects; however, I am having trouble when used in a mpi environment. Please see the following minimal example; works if executed as 'python3 test.py' and fails as 'mpirun -n 4 test.py':

##test.py
import time
from alive_progress import alive_bar
from mpi4py import MPI

def show_progress():                            
    with alive_bar(5000) as progress:                                
        for i in range(5000):                                
            time.sleep(0.001)                                
            progress()                                

if __name__ == '__main__':                                
    if MPI.COMM_WORLD.Get_rank() == 0:                                
        show_progress()    ## This line does not output to the terminal
        show_bars()        ## This line works and outputs as expected

[rsalmei]

Hey @alentner, thank you.

Please, give me some context, what is an "mpi environment"? And this mpirun, what does it do?

Well, I'd guess it is not reporting itself as tty enabled. You could try passing force_tty=True, this will force alive_progress to render in any situation. But please explain to me the mpi thing...

[alentner]

@rsalmei,

The mpi thing ...

MPI (Message Passing Interface) is a standard for developming portable message-passing programs in C, C++, and Fortran. Specifically it is widely used to develop and run SIMD (single instruction multiple data) parallel programs for HPC (high performance computing) and other computationally intensive applications.

An MPI program is launched by 'mpirun' where options such as the number of parallel processes desired (e.g., -n [int]) is specified. The MPI implementation being used (e.g., openMPI) then sets up the appropriate communications (e.g., the TCP/IP network stack or an Infiniband implementation) and launches the application code in parallel.

In this case, there is a python package which supports using the MPI library from within python, mpi4pi.

The confusing bit ...

I had a very similar thought that the python interpreter when launched by the MPI runtime does not register the stdout or stderr as a proper tty. I have tested the example above passing force_tty=True as show below.

What is really perplexing to me is that the show_bars() method properly outputs to the terminal but the alive_bar does not, when run with mpirun -n 1 python test.py

#test.py
import time
from alive_progress import alive_bar, show_bars
from mpi4py import MPI

def show_progress():
    with alive_bar(5000, force_tty=True) as progress:
        for i in range(5000):
            time.sleep(0.001)
            progress()

if __name__ == '__main__':
    if MPI.COMM_WORLD.Get_rank() == 0:
        show_progress()    ## This line does not output to the terminal
        show_bars()        ## This line works and outputs as expected

Please let me know if I can further assist in any way. The other students in my research group very much appreciate the useful and visually appealing progress information I am able to provide in my projects by using alive-progress! I detest having to revert back to simple text driven messages and logging for execution in parallel (spoiled by all those 'astonishing animations')

Thank you, Aaron

[rsalmei]

Hey @alentner, I'm very sorry I forgot to reply to you! But I did read and dig very much your explanation of MPI, thanks!

Of course now I need to read it again... But I will soon, ok? Just to say this won't be ignored 😉

[alentner]

Hey @rsalmei, I very much like the updates to alive-progress!!!

I have checked back on this simple example to see if version 3 suffers from similar issues. This is what I found (Bottom line -- I mostly works now!).

Using an updated minimal example:

import time
from alive_progress import alive_bar
from alive_progress.styles.exhibit import showtime
from mpi4py import MPI

def show_progress():
    with alive_bar(5000, force_tty=True) as progress:
        ## The force_tty option is needed under mpi
        for i in range(5000):
            time.sleep(0.001)
            progress()

if __name__ == '__main__':
    if MPI.COMM_WORLD.Get_rank() == 0:
        show_progress()    ## This line outputs to the terminal w/ clipping
        showtime()         ## This line does not work (i.e., says it needs tty)

What is now perplexing to me is that there is output to the terminal but the progress bar clips the last several characters when run under an MPI enviornment.

For example:

mpirun -n 1 python test.py
|████████████████████▌                   | ▃▁▃ 2565/5000 [51%] in 3s (~3s, 928.1

python test.py
|█████████████████████████▍              | ▅▇▇ 3179/5000 [64%] in 3s (~2s, 930.5/s)

Finally, as noted above, there does not seem to be an option to provide showtime() with a force_tty, this does not seem necessary and I would not see any use case for showcasing the styles under an MPI environment; it failed with the following traceback message: UserWarning: This must be run on a tty connected terminal.

Anyway, thanks for all the hard work, it looks great. If you have any ideas why the last few characters would be clipped, that would be appreciated.

Thanks, Aaron