Open abouteiller opened 5 years ago
Original comment by Aurelien Bouteiller (Bitbucket: abouteiller, GitHub: abouteiller).
Trying to replicate
Original comment by Kai Keller (Bitbucket: kellekai, GitHub: kellekai).
Hi Nuria,
yes I do. I execute with:
mpirun --oversubscribe --mca btl tcp,self --mca mpi_ft_detector_period 1e-1 --mca mpi_ft_detector_timeout 3e-1 -n 2
can you reproduce it?
I noticed that routine ompi_request_state_ok
in file req_ft.c returns with true, maybe it shouldn’t?
Original comment by Aurelien Bouteiller (Bitbucket: abouteiller, GitHub: abouteiller).
Kai,
Thank you for your report. Please try again with a version later than 0e249ca1ae5cb27a3f3d907173b65db188380ce5; it resolves a number of deadlocks scenarios and should address your problem.
Original report by Kai Keller (Bitbucket: kellekai, GitHub: kellekai).
a simple case is for 2 processes. Process 1 fails and process 2 calls MPI_Recv, the result is a deadlock. The same happens for MPI_Allreduce, when some processes fail after all not failing processes have called MPI_Allreduce.
Here a simple example for the first case:
another example for the second case: