search

Normaliz / Normaliz.jl

https://normaliz.github.io/Normaliz.jl/
4 stars 2 forks source link

Fails to build #39

Closed thchr closed 7 months ago

thchr commented 8 months ago

I noticed that there had been quite a few updates to this repository and wanted to see if I could load the library now. Unfortunately, trying to add the project fails on precompilation/build:

julia> versioninfo()
Commit 3120989f39b (2023-12-25 18:01 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 8 × 11th Gen Intel(R) Core(TM) i5-1145G7 @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, tigerlake)
  Threads: 1 on 8 virtual cores

(@v1.10) pkg> add https://github.com/Normaliz/Normaliz.jl
     Cloning git-repo `https://github.com/Normaliz/Normaliz.jl`
    Updating git-repo `https://github.com/Normaliz/Normaliz.jl`
   Resolving package versions...
   Installed normaliz_jll ───────── v300.900.301+0
   Installed Arb_jll ────────────── v200.2300.0+0
   Installed Antic_jll ──────────── v0.201.500+0
   Installed nauty_jll ──────────── v2.6.13+1
   Installed FLINT_jll ──────────── v200.900.9+0
   Installed Calcium_jll ────────── v0.401.100+0
   Installed CMake_jll ──────────── v3.28.1+0
   Installed libcxxwrap_julia_jll ─ v0.12.2+0
   Installed Nemo ───────────────── v0.43.1
   Installed CxxWrap ────────────── v0.15.0
   Installed RandomExtensions ───── v0.4.4
   Installed AbstractAlgebra ────── v0.40.1
  Downloaded artifact: Antic
  Downloaded artifact: FLINT
  Downloaded artifact: nauty
  Downloaded artifact: CMake
  Downloaded artifact: Calcium
  Downloaded artifact: Arb
  Downloaded artifact: normaliz
  Downloaded artifact: libcxxwrap_julia
    Updating `~/.julia/environments/v1.10/Project.toml`
  [0c79eb54] + Normaliz v0.1.0 `https://github.com/Normaliz/Normaliz.jl#master`
    Updating `~/.julia/environments/v1.10/Manifest.toml`
  [c3fe647b] + AbstractAlgebra v0.40.1
  [1f15a43c] + CxxWrap v0.15.0
  [2edaba10] + Nemo v0.43.1
  [0c79eb54] + Normaliz v0.1.0 `https://github.com/Normaliz/Normaliz.jl#master`
  [fb686558] + RandomExtensions v0.4.4
  [e21ec000] + Antic_jll v0.201.500+0
  [d9960996] + Arb_jll v200.2300.0+0
  [3f4e10e2] + CMake_jll v3.28.1+0
  [fcfa6d1b] + Calcium_jll v0.401.100+0
⌅ [e134572f] + FLINT_jll v200.900.9+0
  [1d63c593] + LLVMOpenMP_jll v15.0.7+0
⌅ [656ef2d0] + OpenBLAS32_jll v0.3.24+0
  [3eaa8342] + libcxxwrap_julia_jll v0.12.2+0
  [55c6dc9b] + nauty_jll v2.6.13+1
⌃ [6690c6e9] + normaliz_jll v300.900.301+0
  [3a97d323] + MPFR_jll v4.2.0+1
        Info Packages marked with ⌃ and ⌅ have new versions available. Those with ⌃ may be upgradable, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use `status --outdated -m`
    Building Normaliz → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/8e60b5d8698f25fa9c52619802fac2a82b4afbe6/build.log`
ERROR: Error building `Normaliz`:
CMake Warning (dev) at CMakeLists.txt:1 (project):
  cmake_minimum_required() should be called prior to this top-level project()
  call.  Please see the cmake-commands(7) manual for usage documentation of
  both commands.
This warning is for project developers.  Use -Wno-dev to suppress it.

-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Deprecation Warning at CMakeLists.txt:3 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

-- Found Julia executable: /home/tchr/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/bin/julia
-- Julia_VERSION_STRING: 1.10.0
-- Julia_INCLUDE_DIRS:   /home/tchr/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/include/julia
-- Julia_LIBRARY_DIR:    /home/tchr/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/lib
-- Julia_LIBRARY:        /home/tchr/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/lib/libjulia.so.1.10
-- JULIA_HOME:           /home/tchr/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/bin
-- Julia_LLVM_VERSION:   v15.0.7
-- Julia_WORD_SIZE:      64
-- Found Julia: /home/tchr/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/lib/libjulia.so.1.10 (found version "1.10.0")
-- Found JlCxx at /home/tchr/.julia/artifacts/5209ca23f516fb3391b885eef717e49b4ee0a268/lib
-- Configuring done (2.0s)
-- Generating done (0.0s)
-- Build files have been written to: /home/tchr/.julia/packages/Normaliz/aJsQn/deps/src/build
[ 50%] Building CXX object CMakeFiles/normaliz_julia.dir/normaliz.cpp.o
[100%] Linking CXX shared library lib/libnormaliz_julia.so
[100%] Built target normaliz_julia
ERROR: LoadError: IOError: open("/home/tchr/.julia/packages/Normaliz/aJsQn/deps/../src/Normaliz.jl", 65, 438): permission denied (EACCES)
Stacktrace:
 [1] uv_error
   @ Base ./libuv.jl:100 [inlined]
 [2] open(path::String, flags::UInt16, mode::UInt16)
   @ Base.Filesystem ./filesystem.jl:121
 [3] touch(path::String)
   @ Base.Filesystem ./file.jl:458
 [4] top-level scope
   @ ~/.julia/packages/Normaliz/aJsQn/deps/build.jl:77
 [5] include(fname::String)
   @ Base.MainInclude ./client.jl:489
 [6] top-level scope
   @ none:5
in expression starting at /home/tchr/.julia/packages/Normaliz/aJsQn/deps/build.jl:77

I imagine there's not an abundance of developer time to fix this, but I figured I'd note it in anyway - maybe there is time one day.

Also fails on Windows, but I imagined that's expected ```jl julia> versioninfo() Julia Version 1.10.0 Commit 3120989f39 (2023-12-25 18:01 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Windows (x86_64-w64-mingw32) CPU: 8 × 11th Gen Intel(R) Core(TM) i5-1145G7 @ 2.60GHz WORD_SIZE: 64 LIBM: libopenlibm LLVM: libLLVM-15.0.7 (ORCJIT, tigerlake) Threads: 1 on 8 virtual cores Environment: JULIA_EDITOR = code.cmd JULIA_PKG_PRESERVE_TIERED_INSTALLED = true (@v1.10) pkg> add https://github.com/Normaliz/Normaliz.jl Cloning git-repo `https://github.com/Normaliz/Normaliz.jl` Updating git-repo `https://github.com/Normaliz/Normaliz.jl` Updating registry at `C:\Users\tchr\.julia\registries\General.toml` Resolving package versions... Installed CMake_jll ──────────── v3.28.1+0 Installed normaliz_jll ───────── v300.900.301+0 Installed FLINT_jll ──────────── v200.900.9+0 Installed nauty_jll ──────────── v2.6.13+1 Installed OpenBLAS32_jll ─────── v0.3.24+0 Installed libcxxwrap_julia_jll ─ v0.12.2+0 Installed RandomExtensions ───── v0.4.4 Installed CxxWrap ────────────── v0.15.0 Installed Nemo ───────────────── v0.43.1 Installed AbstractAlgebra ────── v0.40.2 Downloaded artifact: FLINT Downloaded artifact: OpenBLAS32 Downloaded artifact: CMake Downloaded artifact: libcxxwrap_julia Updating `C:\Users\tchr\.julia\environments\v1.10\Project.toml` [0c79eb54] + Normaliz v0.1.0 `https://github.com/Normaliz/Normaliz.jl#master` Updating `C:\Users\tchr\.julia\environments\v1.10\Manifest.toml` [c3fe647b] + AbstractAlgebra v0.40.2 [1f15a43c] + CxxWrap v0.15.0 [2edaba10] + Nemo v0.43.1 [0c79eb54] + Normaliz v0.1.0 `https://github.com/Normaliz/Normaliz.jl#master` [fb686558] + RandomExtensions v0.4.4 [e21ec000] + Antic_jll v0.201.500+0 [d9960996] + Arb_jll v200.2300.0+0 [3f4e10e2] + CMake_jll v3.28.1+0 [fcfa6d1b] + Calcium_jll v0.401.100+0 ⌅ [e134572f] + FLINT_jll v200.900.9+0 ⌅ [656ef2d0] + OpenBLAS32_jll v0.3.24+0 [3eaa8342] + libcxxwrap_julia_jll v0.12.2+0 [55c6dc9b] + nauty_jll v2.6.13+1 ⌃ [6690c6e9] + normaliz_jll v300.900.301+0 [3a97d323] + MPFR_jll v4.2.0+1 Info Packages marked with ⌃ and ⌅ have new versions available. Those with ⌃ may be upgradable, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use `status --outdated -m` Building Normaliz → `C:\Users\tchr\.julia\scratchspaces\44cfe95a-1eb2-52ea-b672-e2afdf69b78f\8e60b5d8698f25fa9c52619802fac2a82b4afbe6\build.log` ERROR: Error building `Normaliz`: ERROR: LoadError: UndefVarError: `CompilerSupportLibraries_jll` not defined Stacktrace: [1] top-level scope @ C:\Users\tchr\.julia\packages\Normaliz\aJsQn\deps\build.jl:43 [2] include(fname::String) @ Base.MainInclude .\client.jl:489 [3] top-level scope @ none:5 in expression starting at C:\Users\tchr\.julia\packages\Normaliz\aJsQn\deps\build.jl:39 ```
w-bruns commented 8 months ago

Last spring Max Horn and I pushed the interface a bit forward. We got stuck when we tried to integrate e-antic for algebraic polytopes because of a complicated Boost pointer type that we could not right away map to julia. I think Max has been very busy since then in the Oscar project. Actually Normaliz is a julia artefact in the Oscar project and used by polymake, one of the big building blocks of Oscar. for which there is now an official release. I think there are three possible ways out:

I will take up the issue again and explore what can be done.

fingolfin commented 7 months ago

The idea here was and is to not build Normaliz on the fly (except for development purposes) but rather to rely on prebuilt binaries provided via the excellent Julia JLL system. When @w-bruns writes "Normaliz is a julia artefact in the Oscar project" then this really refers to normaliz_jll which we indeed originally created for use in OSCAR, but which this package now also uses, and which in principle any Julia package can use. Its source is here: https://github.com/JuliaPackaging/Yggdrasil/tree/master/N/normaliz

Using normaliz_jll is what I implemented last year, when @w-bruns was visiting. The one drawback of this is that to update to a newer normaliz, or to change build options for normaliz, that JLL must also be updated. The big advantage is that it removes a ton of ways things can go wrong, and also avoids lengthy compilation times.

However, that JLL is currently not built against e-antic. There is no fundamental reason for that other than that to change that, one probably would also have to create a JLL for e-antic, and I didn't have time for that (and frankly e-antic support is not an immediate priority for me personally, and my spare time sadly is very limited these days).

But building it against e-antic is only half of the work needed -- the other work is to extend the C++ glue code to deal with it...

This glue code we need to call into the C++ code is in deps/src and based on the Julia package CxxWrap. That code is still compiled on the fly -- on the long run however, it would be better to also provide a JLL for that (just as do for Singular.jl and Polymake.jl -- I could go into detailsl if someone wants to know, just ask). Again this not rocket science, I know in principle all steps needed for this -- rather it's a matter of finding the time to do it (as you guessed).

I'll have a quick look at the build issues now (this was last tested when Julia 1.8 was current), but I can't promise a quick fix (but I'll try to at least post my finding here)

fingolfin commented 7 months ago

Ah it seems I neglected to add compat bounds for CxxWrap so it now uses a new incompatible version.

w-bruns commented 7 months ago

Please forgive my ignorance of the Julia mechanics. Any way, I am happy that things go on. e-antic is not so important that it is worth waiting for it.

fingolfin commented 7 months ago

Partially resolved with PR #41 -- it still fails in a doctest on macOS (though it passes the regular tests), otherwise seems to be OK.

thchr commented 7 months ago

First, thanks for looking at this @fingolfin..! I'm aware of the .jll approach taken here but do not understand the details well enough to really debug errors.

I tried checking out the repo again after #41. The situation is improved (e.g., now builds normaliz_julia) but it still ends up throwing when hitting the touch(joinpath(...)) line at the end of deps/build.jl:

(@v1.10) pkg> add https://github.com/Normaliz/Normaliz.jl
[...]
    Building Normaliz → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/5323d41da8530e634fef19a9697b3a422e9c44ac/build.log`
ERROR: Error building `Normaliz`:
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Julia executable: /home/tchr/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/bin/julia
-- Julia_VERSION_STRING: 1.10.2
-- Julia_INCLUDE_DIRS:   /home/tchr/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/include/julia
-- Julia_LIBRARY_DIR:    /home/tchr/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/lib
-- Julia_LIBRARY:        /home/tchr/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/lib/libjulia.so.1.10
-- JULIA_HOME:           /home/tchr/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/bin
-- Julia_LLVM_VERSION:   v15.0.7
-- Julia_WORD_SIZE:      64
-- Found Julia: /home/tchr/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/lib/libjulia.so.1.10 (found version "1.10.2")
-- Found JlCxx at /home/tchr/.julia/artifacts/fb412eee87eae845b84a799f0cabf241142406d7/lib
-- Configuring done (2.2s)
-- Generating done (0.0s)
-- Build files have been written to: /home/tchr/.julia/packages/Normaliz/MtRb2/deps/src/build
[ 50%] Building CXX object CMakeFiles/normaliz_julia.dir/normaliz.cpp.o
[100%] Linking CXX shared library lib/libnormaliz_julia.so
[100%] Built target normaliz_julia
ERROR: LoadError: IOError: open("/home/tchr/.julia/packages/Normaliz/MtRb2/deps/../src/Normaliz.jl", 65, 438): permission denied (EACCES)
Stacktrace:
 [1] uv_error
   @ ./libuv.jl:100 [inlined]
 [2] open(path::String, flags::UInt16, mode::UInt16)
   @ Base.Filesystem ./filesystem.jl:121
 [3] touch(path::String)
   @ Base.Filesystem ./file.jl:458
 [4] top-level scope
   @ ~/.julia/packages/Normaliz/MtRb2/deps/build.jl:77
 [5] include(fname::String)
   @ Base.MainInclude ./client.jl:489
 [6] top-level scope
   @ none:5
in expression starting at /home/tchr/.julia/packages/Normaliz/MtRb2/deps/build.jl:77

What is the purpose of that line? I haven't seen similar things elsewhere.

Version info This is on Ubuntu. On Windows, I get a different error: ```jl ERROR: Error building `Normaliz`: ERROR: LoadError: UndefVarError: `CompilerSupportLibraries_jll` not defined Stacktrace: [1] top-level scope @ C:\Users\tchr\.julia\packages\Normaliz\MtRb2\deps\build.jl:43 [2] include(fname::String) @ Base.MainInclude .\client.jl:489 [3] top-level scope @ none:5 in expression starting at C:\Users\tchr\.julia\packages\Normaliz\MtRb2\deps\build.jl:39 ```
thchr commented 7 months ago

Scratch that: it seems the error was due to it still somehow pulling an old version. After a rm and fresh add everything worked out perfectly on Ubuntu.

I'll go ahead and close this then: resolved!

fingolfin commented 7 months ago

Well that touch still is problematic. It was/is a bit of a hack to force a fresh precompilation run if libnormaliz_julia is rebuilt. However, since then, I've got a patch merged into CxxWrap that makes it do that automatically, so I can get rid of that touch now, I think (to be verified).