Closed leofang closed 3 years ago
FYI: the closest (but still not directly relevant) discussion I can find is this: https://developercommunity.visualstudio.com/content/problem/446886/cannot-uninstall-microsoft-mpi-in-azure-pipeline-o.html
An alternative strategy to test the package before releasing it is to enable the build artifact to be stored by Azure via adding this config
azure:
store_build_artifacts: true
to conda-forge.yml
so that it can be downloaded and tested locally.
Hi @mariusvniekerk Matt told me you might share some insight with us on Azure's infrastructure 😅 Any chance you have a clue off top of your head for why we can't have a matching PMP version?
In #6 I added a hacky solution to this problem: Force-installing the MS-MPI Runtime to overwrite the existing one in the vm image: https://github.com/conda-forge/msmpi-feedstock/blob/4a424b89b0a79c928d9902c2082a6548ef86e1ca/recipe/bld.bat#L38-L44 All MPI tests can run afterwards, but this is just a very bad band-aid fix that only applies to the current repo and not to all downstream repos. However, this is the best I can do.
@leofang Just a clarification. I believe this issue does not affect Azure's vmImage: 'windows-2019'
. I'm using that image for mpi4py's CI, I'm installing latest MS MPI from official installers, and I have no issues.
Trust me, @dalcinl. See all the recent failing attempts in https://github.com/conda-forge/msmpi-feedstock/pull/7. In mpi4py's CI, it's precisely what we do now in the msmpi
and mpi4py
conda packages: We force reinstall a fresh, new MS-MPI Runtime to overwrite the existing MS-MPI v7, so it works.
@conda-forge/core do you all have any ideas here on how to help fix this?
Gitter discussion starts at https://gitter.im/conda-forge/conda-forge.github.io?at=608837fad24b29049788a9e1
Can you show the full debug message?
@isuruf check the last failing CI output in #7.
Do you have a link for a msmpi version that's built from source? The debug output is different from what I see on source posted in github.
Unfortunately Azure has removed all failing tests (note that this issue was created back in January). But I can assure you the SMP version mismatch (request: received version 3, expected 4
) happens with both the source build and the binary installer.
@leofang if you can open a new PR with the failure, that'd help others debug more easily.
I reopened #7 as a playground for you.
So #11 fixed multiple issues. It includes a patch -- which is the missing key that I didn't realize when I asked for help on Gitter -- to allow downstream packages to test msmpi
out of box:
https://github.com/conda-forge/msmpi-feedstock/blob/7fc2c26641837eb233879473ca3ee0dd27242edb/recipe/activate.bat#L5-L12
I just verified with mpi4py that it's working:
https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=310547&view=results
This patch is only applied when building in the CI.
Deleting of those DLLs should be moved to https://github.com/conda-forge/conda-forge-ci-setup-feedstock/blob/master/recipe/run_conda_forge_build_setup_win.bat#L53.
To avoid this issue on user systems, we should rename the msmpi.dll to msmpi-10.dll
and fix msmpi.lib
to use that name.
As title. Below I am summarizing and leaving a note for Conda-Forge downstream recipe maintainers on my struggle starting in https://github.com/conda-forge/mpi4py-feedstock/pull/33#issuecomment-755181656.
Do not test this in your Win64 builds:
what you'll get is the following error:
If you run
mpiexec
with the full debug log on (mpiexec -d 3 ...
) you'd see that a wrong PMP version is picked up:But the reason is currently unknown. The PMP version from MS MPI v10 (built in this feedstock) is 4.
My best theory is there is a hidden conflict between Conda-Forge's CI setup vs Azure's infrastructure. It could be that there's an
smpd
-like orchestrating daemon running in the background which got picked up to talk tompiexec
, so the MPI processes cannot pass the version check. (Ideally, both thesmpd
manager and all MPI processes would agree upon the PMP version.)Note that, although not listed on the CI image (vs2017-win2016)'s software list, the image contains an outdated MS MPI (v7.x) in
C:\Program Files\Microsoft MPI\
. I tried force-removing it before building & testing, but it didn't fix the error. So for those who are interested in hunting down where the PMP version 3 comes from, you should first investigate if it's due toC:\Program Files\Microsoft MPI\Bin\smpd.exe
being launched in the background when the CI starts. I am not familiar with Windows, so after confirming the build is correct I have 0 interest in digging deeper...cc: @conda-forge/core for the record.