MS-MPI hangs in MPI_Finalize() inside DLL.

Hi all.

I have a case where I'm using MPI inside a DLL file. It serves like an API which means that MPI_Finalize() must be called only once, from inside the DLL, in the end of its caller program. To accomplish this I've put MPI_Finalize() in a function that is marked with __attribute__((destructor)), so its code will execute upon unloading of the DLL.

However, the program hangs there, and MPI_Finalize() never returns. My setup is (although the issue was reproducible in other PC as well): Windows 10 Pro Version 21H1 (OS Build 19043.1237). MS-MPI Version 10.1.12498.18. Build with gcc 10.1.0 using mingw64 from MSYS2 with target x86_64-w64-mingw32.

I have managed to reproduce the issue in a minimal example:

myapi.cpp (The DLL code)


#include <stdio.h>
#include <stdint.h>
#include <mpi.h>

void bin_destructor() attribute((destructor));

void bin_destructor(){ int initialized; MPI_Initialized(&initialized); int finalized; MPI_Finalized(&finalized);

printf(" %s. Line %d \n", FILE, LINE); fflush(stdout); if(initialized && (!finalized)){ printf(" %s. Line %d, Finalizing MPI...", FILE, LINE); fflush(stdout); MPI_Finalize(); // <-- It hangs here printf("Done. \n"); fflush(stdout); } printf(" %s. Line %d \n", FILE, LINE); fflush(stdout); }

extern "C" stdcall declspec(dllexport) int MyAPI(const int n){ int initialized; MPI_Initialized(&initialized); if( !initialized ) MPI_Init(NULL, NULL); // Initialize only once. printf(" %s. Line %d n=%d \n", FILE, LINE, n); fflush(stdout);

return 0; }

compiling with
```shell
g++ -shared -o libmyapi.dll -Wall -fPIC -IC:\SourceCode\MPI\Include myapi.cpp C:\Windows\System32\msmpi.dll

main.cpp (Program that uses the DLL)


#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>

extern "C" int MyAPI(const int n);

int main(int argc, char argv){ printf("** %s. Line %d \n", FILE, LINE); fflush(stdout); MyAPI(1); printf(" %s. Line %d \n", FILE, LINE); fflush(stdout); MyAPI(2); printf(" %s. Line %d \n", FILE, LINE); fflush(stdout);

return 0; }

compiling with
```script
g++ -o main.exe -Wall main.cpp -L. -llibmyapi

When I run the program serially, e.g. $>main.exe , the output is:

*** main.cpp. Line 8 ***
*** myapi.cpp. Line 26 n=1 ***
*** main.cpp. Line 10 ***
*** myapi.cpp. Line 26 n=2 ***
*** main.cpp. Line 12 ***
*** myapi.cpp. Line 13 ***
*** myapi.cpp. Line 15, Finalizing MPI...Done. ***
*** myapi.cpp. Line 19 ***

When I run the program in parallel, even with only 1 process, e.g. $>mpiexec -np 1 main.exe , the output is:

*** main.cpp. Line 8 ***
*** myapi.cpp. Line 26 n=1 ***
*** main.cpp. Line 10 ***
*** myapi.cpp. Line 26 n=2 ***
*** main.cpp. Line 12 ***
*** myapi.cpp. Line 13 ***
*** myapi.cpp. Line 15, Finalizing MPI...

..and it hangs there.

Any help so I can explain & resolve this behavior is greatly appreciated. Best regards, George

microsoft / Microsoft-MPI

MS-MPI hangs in MPI_Finalize() inside DLL. #59