search

open-mpi / hwloc

Hardware locality (hwloc)
https://www.open-mpi.org/projects/hwloc
Other
564 stars 173 forks source link

cmake support for native Windows build #88

Closed ompiteam closed 2 years ago

ompiteam commented 10 years ago

Shiqing (former OMPI windows builder) sent a draft of patch (a couple months ago) to enable native Windows build with cmake.

ompiteam commented 10 years ago

Imported from trac issue 87. Created by bgoglin on 2012-11-11T06:13:24, last modified: 2013-03-06T09:23:12

ompiteam commented 10 years ago

Trac comment by bgoglin on 2013-03-06 09:23:12:

The new ci.inria.fr/hwloc continous integration service builds Windows 32bits and 64bits winballs automatically every night, and it can be used to build release winballs very easily too. No need to play with cmake very soon.

emmenlau commented 2 years ago

I'd kindly like to bump up this thread. I tried to build hwloc on Windows with MSVC and also with Clang-Cl compilers. This was not particularly easy using the provided Visual Studio project files.

However I could find that the (well-known) vcpkg project has support for hwloc with cmake. They maintain a full-blown CMakeLists.txt at https://github.com/microsoft/vcpkg/tree/master/ports/hwloc/.

I used this CMakeLists.txt with the latest official release 2.5.0, and needed to add three newer source files. Afterwards the build worked like a charm. CMake makes it quite easy to experiment with different settings and different compilers, particularly on Windows where the support for scripted building is a bit lacking.

It would be awesome if similar support would be in hwloc directly, not in external projects that maintain their own, hand-crafted solutions. Would it be possible to adopt this CMakeLists.txt?

bgoglin commented 2 years ago

If this code is indeed under MIT licence (found at the root of the repo) and redistributing MIT-license code under our BSD-licence is OK, then that's indeed an easy solution that could go under contrib and be easily tested by our CI. Do you only need the library or do you need multiple tools too? To be clear, we could try building some tools, but we won't enable tests, etc. Also I need to understand why they are replacing some things in the windows config.h that we provide. I need to check when snprintf and strtoull are available on Windows.

emmenlau commented 2 years ago

Thanks a lot @bgoglin for the positive response!

I'm under the impression that the license (MIT) applies here and would allow adopting the code. Furthermore, I'm under the impression that its in the best interest of vcpkg that more and more upstream projects adopt cmake. At least in my typical work, more than 90% of projects nowadays have native cmake support or at least provide cmake build instructions in a contrib section.

For my needs, I use hwloc as a prerequisite of oneTBB. Therefore I only need the core library, no tools. oneTBB provides a cmake build, so I think that many users of oneTBB would be happy if hwloc would provide (at least basic) cmake support.

I also saw that vcpkg is replacing some definitions in the windows config.h. There is a corresponding code block in https://github.com/microsoft/vcpkg/blob/a79d4a8ec7ae4464eb22f6af23e277882a67595d/ports/hwloc/CMakeLists.txt#L10. Furthermore they make some changes again in https://github.com/microsoft/vcpkg/blob/a79d4a8ec7ae4464eb22f6af23e277882a67595d/ports/hwloc/portfile.cmake#L19. Both files are written in cmake syntax. Note that this refers to hwloc 2.2.0, and possibly the changes are just not needed anymore in newer hwloc?

I did apply the changes from CMakeLists.txt, but not the ones from portfile.cmake, and can confirm that I got a working shared library build when building with Visual Studio 2019 v16.11.5 and also with ClangCl 13.0.0 from upstream LLVM on Windows.

bgoglin commented 2 years ago

Can you clarify the difference between CMakeLists.txt and portfile.cmake? Any idea why the changes applied in portfile.cmake after cmake install are not applied directly inside CMakeLists.txt? Since adding this stuff isn't going to break anything, I am going to include it in hwloc 2.6 (to be released very soon). We'll get some early testing and fix CMakeLists.txt in 2.6.1 or 2.7 if needed.

bgoglin commented 2 years ago

For snprintf(), it had broken return values in some older versions, and we have no way to detect that at build time. For strtoull, it was enabled on Windows until 2016 but somebody had problems and made it use _strtoui64() instead. I am tempted to ignore those changes from CMakeLists.txt since it shouldn't hurt.

emmenlau commented 2 years ago

Can you clarify the difference between CMakeLists.txt and portfile.cmake? Any idea why the changes applied in portfile.cmake after cmake install are not applied directly inside CMakeLists.txt?

The portfile.cmake contains the actual build instructions that are used by vcpkg. Its implemented in cmake. But it will execute a separate call to cmake to actually build the software. Why they chose to split these editing commands into two files is beyond me. I can only speculate that the portfile may run its code "later", that is that something is happening between executing the build (using CMakeLists.txt) and the later code in portfile.cmake. It may be just easier to add this in portfile.cmake rather than as a post-build-step in CMakeLists.txt (albeit both is possible).

bgoglin commented 2 years ago

Can you try https://github.com/bgoglin/hwloc/blob/windows-cmake/contrib/windows-cmake/CMakeLists.txt ? It's updated for hwloc 2.4 or later. It builds all tools and lstopo by default (can be disabled with HWLOC_SKIP_TOOLS and HWLOC_SKIP_LSTOPO). 2 removals from vcpkg: strtoull and snprintf as explained earlier, and tools go to "bin/" instead of "tools/hwloc/". I am still working on testing it in our CI but it should be ready very soon.

emmenlau commented 2 years ago

Awesome! I'll try to add this to our CI sometime later today!