search

OpenMS / OpenMS

The codebase of the OpenMS project
https://www.openms.de
Other
477 stars 314 forks source link

vcpkg might have more requirements than our contrib #7224

Open timosachsenberg opened 10 months ago

timosachsenberg commented 10 months ago

Not 100% sure why fortran is not found. Maybe not everything is needed and can be restricted? Any thoughts @jpfeuffer 

Building openblas:x64-linux...
-- Downloading https://github.com/xianyi/OpenBLAS/archive/v0.3.25.tar.gz -> xianyi-OpenBLAS-v0.3.25.tar.gz...
-- Extracting source /ceph/ibmi/abi/projects/sachsenb/vcpkg_test/vcpkg/downloads/xianyi-OpenBLAS-v0.3.25.tar.gz
-- Applying patch uwp.patch
-- Applying patch fix-redefinition-function.patch
-- Applying patch install-tools.patch
-- Using source at /ceph/ibmi/abi/projects/sachsenb/vcpkg_test/vcpkg/buildtrees/openblas/src/v0.3.25-dc6d649124.clean
-- Configuring x64-linux
-- Building x64-linux-dbg
-- Building x64-linux-rel
-- Fixing pkgconfig file: /ceph/ibmi/abi/projects/sachsenb/vcpkg_test/vcpkg/packages/openblas_x64-linux/lib/pkgconfig/openblas.pc
-- Fixing pkgconfig file: /ceph/ibmi/abi/projects/sachsenb/vcpkg_test/vcpkg/packages/openblas_x64-linux/debug/lib/pkgconfig/openblas.pc
-- Installing: /ceph/ibmi/abi/projects/sachsenb/vcpkg_test/vcpkg/packages/openblas_x64-linux/share/openblas/copyright
-- Performing post-build validation
Stored binaries in 1 destinations in 1.6 s.
Elapsed time to handle openblas:x64-linux: 2.2 min
Installing 99/136 blas:x64-linux...
Building blas:x64-linux...
-- Installing: /ceph/ibmi/abi/projects/sachsenb/vcpkg_test/vcpkg/packages/blas_x64-linux/share/blas/vcpkg-cmake-wrapper.cmake
-- Configuring x64-linux
-- Performing post-build validation
Stored binaries in 1 destinations in 882 ms.
Elapsed time to handle blas:x64-linux: 8.7 s
Installing 100/136 lapack-reference:x64-linux...
Building lapack-reference[blas-select,core,noblas]:x64-linux...
-- Downloading https://github.com/Reference-LAPACK/lapack/archive/v3.11.0.tar.gz -> Reference-LAPACK-lapack-v3.11.0.tar.gz...
-- Extracting source /ceph/ibmi/abi/projects/sachsenb/vcpkg_test/vcpkg/downloads/Reference-LAPACK-lapack-v3.11.0.tar.gz
-- Applying patch cmake-config.patch
-- Applying patch lapacke.patch
-- Using source at /ceph/ibmi/abi/projects/sachsenb/vcpkg_test/vcpkg/buildtrees/lapack-reference/src/v3.11.0-5e6de67bfb.clean
-- The Fortran compiler identification is unknown
CMake Error at scripts/cmake/vcpkg_find_fortran.cmake:130 (message):
  Unable to find a Fortran compiler using 'CMakeDetermineFortranCompiler'.
  Please install one (e.g.  gfortran) and make it available on the PATH!
Call Stack (most recent call first):
  ports/lapack-reference/portfile.cmake:54 (vcpkg_find_fortran)
  scripts/ports.cmake:170 (include)

error: building lapack-reference:x64-linux failed with: BUILD_FAILED
Elapsed time to handle lapack-reference:x64-linux: 22 s
Please ensure you're using the latest port files with `git pull` and `vcpkg update`.
Then check for known issues at:
  https://github.com/microsoft/vcpkg/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+lapack-reference
You can submit a new issue at:
  https://github.com/microsoft/vcpkg/issues/new?title=[lapack-reference]+Build+error+on+x64-linux&body=Copy+issue+body+from+%2Fceph%2Fibmi%2Fabi%2Fprojects%2Fsachsenb%2Fvcpkg_test%2FOpenMS%2Fvcpkg_installed%2Fvcpkg%2Fissue_body.md
timosachsenberg commented 10 months ago

ahh there might be a noblas option...

/ceph/ibmi/abi/projects/sachsenb/vcpkg_test/OpenMS$ vcpkg depend-info coin-or-cgl
vcpkg-cmake-config:
pthreads: vcpkg-cmake-config
pthread: pthreads
vcpkg-cmake:
openblas: pthread, vcpkg-cmake, vcpkg-cmake-config
blas: openblas, vcpkg-cmake
lapack-reference[noblas, blas-select]: blas, vcpkg-cmake, vcpkg-cmake-config
bzip2[tool]: vcpkg-cmake
coin-or-buildtools:
lapack: lapack-reference, vcpkg-cmake
zlib: vcpkg-cmake
coinutils: bzip2, coin-or-buildtools, lapack, zlib
coin-or-osi: coinutils
coin-or-clp: coin-or-osi, coinutils
coin-or-cgl: coin-or-clp, coin-or-osi, coinutils
jpfeuffer commented 10 months ago

Let's hope.. otherwise changing the coin or recipe is a huge pain. It's one of the worst build systems I have ever seen. Other coin packages will probably need similar treatment or you need to transitively propagate options because all the packages are somehow interdependent.

timosachsenberg commented 9 months ago

Do you know feasible/how much work it would be to go the following route to get rid of contrib:

  1. inlcude vcpkg as a submodule (as most other projects do it)
  2. prefer system packages over building vcpkg files for faster CI. I read that this is possible with port overlays

wouldn't that (potentially) easy dependency management a lot (while still preserving the possibility to build everything from source through vcpkg)=

jpfeuffer commented 9 months ago
  1. should be very easy
  2. not so sure. Packages often have specific dependencies between each other (e.g., if one was built with zlib support, the other needs it, too, etc.). This might fall apart if you allow mixed things. I think I would rather go for caching or an artifactory instead.
  3. Not necessarily much better, since you will lose control over quick edits for new versions of dependencies. If something breaks, you need to wait for a sometimes lengthy PR on vcpkg and often need to make the PR yourself (especially for the not-so-popular and not-so-well maintained packages like coin)
timosachsenberg commented 9 months ago

the main issue seems to be qt... I think all others could be built on github and cached

jpfeuffer commented 9 months ago

Can it not be built? Or not be cached? What's the problem exactly?

timosachsenberg commented 9 months ago

I think it just takes ages to build. 12h+

jpfeuffer commented 9 months ago

I think you could try with the overlay ports. They look straightforward (basically declaring "empty" ports and forwarding the QT_ROOT env var for qt). It probably works more often than not. Just wanted to point out that ideally you build everything from vcpkg.

timosachsenberg commented 9 months ago

Found some examples for qt6 that work with the qt install action: https://github.com/MerginMaps/mobile-sdk/tree/master/vcpkg-overlay/triplets https://github.com/MerginMaps/mobile-sdk/blob/master/.github/workflows/linux.yml

jpfeuffer commented 9 months ago

Yup looks like that's all. Should work with qt5 as well. Except the version number in the var names ofc ;)

timosachsenberg commented 9 months ago

What about the port files? are these just copies of the ones in vcpkg. e.g in this folder https://github.com/MerginMaps/mobile-sdk/blob/master/vcpkg-overlay/ports/qt6/vcpkg-cmake-wrapper.cmake

jpfeuffer commented 9 months ago

should not be necessary. no idea what they did there. please use empty ports first and see if it fails. https://github.com/microsoft/vcpkg/issues/27574

timosachsenberg commented 9 months ago

k nice... vcpkg found it... and it seems to build :)

cmake /ceph/ibmi/abi/projects/sachsenb/vcpkg_test/OpenMS -DVCPKG_OVERLAY_PORTS=/ceph/ibmi/abi/projects/sachsenb/vcpkg_test/OpenMS/vcpkg-overlays/ports -DCMAKE_TOOLCHAIN_FILE=/ceph/ibmi/abi/projects/sachsenb/vcpkg_test/OpenMS/vcpkg/scripts/buildsystems/vcpkg.cmake

see https://github.com/OpenMS/OpenMS/tree/feature/vcpkg

unfortunately fails with:

[ 29%] Linking CXX shared library ../../lib/libOpenMS.so
/usr/bin/ld: cannot find -lBLAS: No such file or directory
/usr/bin/ld: cannot find -lLAPACK: No such file or directory