Closed mohamed-barakat closed 4 years ago
This is an assertion in CxxWrap code, include/jlcxx/type_conversion.hpp
.
Could it be that CxxWrap was compiled with a different Julia version? Perhaps re-building it via Pkg.build
might help? This is a wild guess, though
Thanks for the suggestion, but no, I get the same error, unfortunately:
(v1.3) pkg> build CxxWrap
Building CxxWrap → `~/.julia/packages/CxxWrap/lDNAy/deps/build.log`
julia> using Oscar
[ Info: Precompiling Oscar [f1435218-dba5-11e9-1e4d-f1a5fab5fc13]
Assertion failed: (has_julia_type<T>()), function value, file /Users/mo/.julia/packages/CxxWrap/lDNAy/deps/usr/lib/cmake/JlCxx/../../../include/jlcxx/type_conversion.hpp, line 601.
signal (6): Abort trap: 6
in expression starting at /Users/mo/.julia/packages/Polymake/kIMyn/src/Polymake.jl:41
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 1562765 (Pool: 1562515; Big: 250); GC: 1
ERROR: LoadError: Failed to precompile Polymake [d720cf60-89b5-51f5-aff5-213f193123e7] to /Users/mo/.julia/compiled/v1.3/Polymake/QizyK_VOjQ4.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1283
[3] _require(::Base.PkgId) at ./loading.jl:1024
[4] require(::Base.PkgId) at ./loading.jl:922
[5] require(::Module, ::Symbol) at ./loading.jl:917
[6] include at ./boot.jl:328 [inlined]
[7] include_relative(::Module, ::String) at ./loading.jl:1105
[8] include(::Module, ::String) at ./Base.jl:31
[9] top-level scope at none:2
[10] eval at ./boot.jl:330 [inlined]
[11] eval(::Expr) at ./client.jl:425
[12] top-level scope at ./none:3
in expression starting at /Users/mo/.julia/packages/Oscar/kRdEf/src/Oscar.jl:33
ERROR: Failed to precompile Oscar [f1435218-dba5-11e9-1e4d-f1a5fab5fc13] to /Users/mo/.julia/compiled/v1.3/Oscar/aJ3Pg_VOjQ4.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1283
[3] _require(::Base.PkgId) at ./loading.jl:1024
[4] require(::Base.PkgId) at ./loading.jl:922
[5] require(::Module, ::Symbol) at ./loading.jl:917
Could you please try running ]build Polymake
as well. The error comes from a CxxWrap header but seems to come when loading the polymake shared library (Polymake.jl:41) via wrapmodule.
Thank you @benlorenz. No, same mistake
(v1.3) pkg> build Polymake
Building CxxWrap ──→ `~/.julia/packages/CxxWrap/lDNAy/deps/build.log`
Building CMake ────→ `~/.julia/packages/CMake/ULbyn/deps/build.log`
Building LoadFlint → `~/.julia/packages/LoadFlint/3t3NS/deps/build.log`
Building Polymake ─→ `~/.julia/packages/Polymake/kIMyn/deps/build.log`
(v1.3) pkg> build Singular
Building LoadFlint → `~/.julia/packages/LoadFlint/3t3NS/deps/build.log`
Building Nemo ─────→ `~/.julia/packages/Nemo/J7Y4Z/deps/build.log`
Building CxxWrap ──→ `~/.julia/packages/CxxWrap/lDNAy/deps/build.log`
Building Singular ─→ `~/.julia/packages/Singular/l3jdY/deps/build.log`
(v1.3) pkg> build Oscar
Building CxxWrap ──→ `~/.julia/packages/CxxWrap/lDNAy/deps/build.log`
Building CMake ────→ `~/.julia/packages/CMake/ULbyn/deps/build.log`
Building LoadFlint → `~/.julia/packages/LoadFlint/3t3NS/deps/build.log`
Building Polymake ─→ `~/.julia/packages/Polymake/kIMyn/deps/build.log`
Building Nemo ─────→ `~/.julia/packages/Nemo/J7Y4Z/deps/build.log`
Building Singular ─→ `~/.julia/packages/Singular/l3jdY/deps/build.log`
julia> using Oscar
[ Info: Precompiling Oscar [f1435218-dba5-11e9-1e4d-f1a5fab5fc13]
Assertion failed: (has_julia_type<T>()), function value, file /Users/mo/.julia/packages/CxxWrap/lDNAy/deps/usr/lib/cmake/JlCxx/../../../include/jlcxx/type_conversion.hpp, line 601.
signal (6): Abort trap: 6
in expression starting at /Users/mo/.julia/packages/Singular/l3jdY/src/LibSingular.jl:5
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 3129025 (Pool: 3128574; Big: 451); GC: 2
ERROR: LoadError: Failed to precompile Singular [bcd08a7b-43d2-5ff7-b6d4-c458787f915c] to /Users/mo/.julia/compiled/v1.3/Singular/9fz0y_VOjQ4.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1283
[3] _require(::Base.PkgId) at ./loading.jl:1024
[4] require(::Base.PkgId) at ./loading.jl:922
[5] require(::Module, ::Symbol) at ./loading.jl:917
[6] include at ./boot.jl:328 [inlined]
[7] include_relative(::Module, ::String) at ./loading.jl:1105
[8] include(::Module, ::String) at ./Base.jl:31
[9] top-level scope at none:2
[10] eval at ./boot.jl:330 [inlined]
[11] eval(::Expr) at ./client.jl:425
[12] top-level scope at ./none:3
in expression starting at /Users/mo/.julia/packages/Oscar/kRdEf/src/Oscar.jl:32
ERROR: Failed to precompile Oscar [f1435218-dba5-11e9-1e4d-f1a5fab5fc13] to /Users/mo/.julia/compiled/v1.3/Oscar/aJ3Pg_VOjQ4.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1283
[3] _require(::Base.PkgId) at ./loading.jl:1024
[4] require(::Base.PkgId) at ./loading.jl:922
[5] require(::Module, ::Symbol) at ./loading.jl:917
Thank you @benlorenz. No, same mistake
Not exactly the same, this time the error comes from @wrapmodule
in LibSingular.jl
line 5.
Edit: looked at the wrong file first...
Assertion failed: (has_julia_type<T>()), function value, file /Users/mo/.julia/packages/CxxWrap/lDNAy/deps/usr/lib/cmake/JlCxx/../../../include/jlcxx/type_conversion.hpp, line 601. signal (6): Abort trap: 6 in expression starting at /Users/mo/.julia/packages/Singular/l3jdY/src/LibSingular.jl:5
Not really sure how to properly debug this, you could try editing
/Users/mo/.julia/packages/CxxWrap/lDNAy/deps/usr/lib/cmake/JlCxx/../../../include/jlcxx/type_conversion.hpp
and adding the following before the assert in line 601:
if (has_julia_type<T>())
std::cerr << "Type " + std::string(typeid(T).name()) + " has Julia type" << std::endl;
else
std::cerr << "Type " + std::string(typeid(T).name()) + " has no Julia type" << std::endl;
Then after rebuilding (at least Polymake and Singular) and using Oscar once more you should see some output with the mangled types that are ok and hopefully also the one that seems to be missing in the julia type map.
Thank you very much @benlorenz.
Not really sure how to properly debug this, you could try editing
/Users/mo/.julia/packages/CxxWrap/lDNAy/deps/usr/lib/cmake/JlCxx/../../../include/jlcxx/type_conversion.hpp
and adding the following before the assert in line 601:if (has_julia_type<T>()) std::cerr << "Type " + std::string(typeid(T).name()) + " has Julia type" << std::endl; else std::cerr << "Type " + std::string(typeid(T).name()) + " has no Julia type" << std::endl;
I now encounter the problem with Singular.jl
, so it is not specific to Polymake.jl
. Furthermore, the latest versions of both packages work on MacOs X:
So the problem seems to be with my MacOs X 10.15.3 (with the latest Xcode) with Julia 1.3 and 1.4.
I added the lines you suggested and got the following error:
julia> using Singular
[ Info: Precompiling Singular [bcd08a7b-43d2-5ff7-b6d4-c458787f915c]
Type NSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEE has no Julia type
Assertion failed: (has_julia_type<T>()), function value, file /Users/mo/.julia/packages/CxxWrap/lDNAy/deps/usr/lib/cmake/JlCxx/../../../include/jlcxx/type_conversion.hpp, line 605.
signal (6): Abort trap: 6
in expression starting at /Users/mo/.julia/packages/Singular/GsN0h/src/LibSingular.jl:5
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 2503 (Pool: 2494; Big: 9); GC: 0
ERROR: Failed to precompile Singular [bcd08a7b-43d2-5ff7-b6d4-c458787f915c] to /Users/mo/.julia/compiled/v1.4/Singular/9fz0y_wHM1S.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
[3] _require(::Base.PkgId) at ./loading.jl:1029
[4] require(::Base.PkgId) at ./loading.jl:927
[5] require(::Module, ::Symbol) at ./loading.jl:922
The missing symbol is just std::string
which should have been added to julia by CxxWrap, so this is quite weird.
I tried with the latest xcode that is available on travis (Xcode 11.3.1 on Mac OS 10.14) but could not reproduce this error:
https://travis-ci.com/github/oscar-system/Polymake.jl/builds/158291943
https://travis-ci.com/github/oscar-system/Singular.jl/builds/158292909
(manually triggered with custom build config and new osx_image)
I will try to have a look where this type should be added to the type map ...
Thank you so much for your help @benlorenz.
I tried with the latest xcode that is available on travis (Xcode 11.3.1 on Mac OS 10.14) but could not reproduce this error: https://travis-ci.com/github/oscar-system/Polymake.jl/builds/158291943 https://travis-ci.com/github/oscar-system/Singular.jl/builds/158292909 (manually triggered with custom build config and new osx_image)
I see. My XCode is Version 11.4 (11E146), maybe this is the problem.
Could the problem be one of a version mismatch between the used C++ libraries? I.e. a breakage in the C++ ABI, which is exacerbated by the fact that we combined precompiled C++ binaries that we download with freshly compiled C++ code, i.e., code that was made with different compilers?
For GCC, here Jonathan Wakely (one of the developers of libstdc++ and member of the ISO C++ working group) says:
Similarly, if you compile one object with GCC 7 and -std=c++17 and another object with GCC 8 and -std=c++17 you will have problems, because C++17 support in GCC 7 and 8 is still experimental and evolving.
Of course on a Mac, the default compiler is clang not GCC, but I'd not be surprised if similar statements hold there.
Also note that C++11 (yes, the standard) introduced an ABI breakage in std::string
to facilitate optimizations that were not possible with the old standard. GCC resp. libstdc++ tries hard to ensure you don't actually get an ABI breakage, by including two implementations of std::string, one with the old semantics/ABI and one with the new.
Looking at the error, I wonder if perhaps somehow the two implementations are both used here... Could it be that part of the C++ code involved here is compiled with -std=c++17
(the CxxWrap 0.9 build system seems to enforce this), but other parts are not / are compiled with an older standard? Just wondering.
(Did I mention that I am really not happy with CxxWrap requiring C++17? Because I am not happy :/)
I don't think this is caused by any ABI change because on MacOS everything uses libc++ (instead of libstdc++) which only supports C++11 (and newer). The full demangled symbol from the error is:
std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >
Here the __1
indicates that it is a libc++ symbol.
The corresponding symbol with libstdc++ would be
std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >
where the __cxx11
namespace indicates that it uses the new C++11 abi which is the default for GCC 6 (GCC 7 on debian) and newer (if remember this correctly). The old abi has this symbol without __cxx11
.
I finally found a way to reproduce and test this (albeit somewhat cumbersome): https://github.com/oscar-system/Polymake.jl/runs/570804489
So this is definitely coming from Xcode 11.4, default on github actions is 11.3.1 and I changed it to 11.4 in runtests.yml
for this testing branch.
So I can finally do some more experiments later today.
Thank @benlorenz for investigating. I am relieved :)
@mohamed-barakat I may have suggested this before, but, just to be sure there is no corruption going on: have you tried temporarily renaming your ~/.julia
directory, then reinstalling Oscar.jl (and thus automatically all of its dependencies) ? If that doesn't help, at least we know it's a systematic issue and not some random corruption.
Yes, when I encountered this problem I even removed ~/.julia
, installed Julia 1.4.0
and tried ] add Oscar
.
It shouldn't be any corruption, I can reproduce this on github actions and it is somehow coming from the typeid(T).hash_code()
that is different if some parts of the running program were built with Xcode 11.4.
My current debug output contains the following:
2020-04-08T20:27:19.3130800Z ** Inserting NSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEE with hash 5163132283 into typemap
...
Here std::string was inserted into the jlcxx type map by code compiled with an old xcode in the libcxxwrap-julia library (fetched via cxxwrap build.jl). Then there are plenty more types inserted, all with hashes in a similar range (i.e. 5 billion).
After a while we can see the polymake types starting in the second line of the following block with N2pm4perl13PropertyValueE
but at the same time the hash changes to a lot larger values:
...
2020-04-08T20:27:19.3217870Z ** Inserting PNSt3__110unique_ptrIaNS_14default_deleteIaEEEE with hash 5163357615 into typemap
2020-04-08T20:27:22.2740970Z ** Inserting N2pm4perl13PropertyValueE with hash 15027391949131603696 into typemap
2020-04-08T20:27:22.2741200Z ** Inserting N5jlcxx10BoxedValueIN2pm4perl13PropertyValueEEE with hash 11147375521081314937 into typemap
...
2020-04-08T20:27:22.2744560Z Type NSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEE with hash 7048117399172095301 has no Julia type
2020-04-08T20:27:22.2745910Z Assertion failed: (has_julia_type<T>()), function value, file /Users/runner/.julia/packages/CxxWrap/lDNAy/deps/../../../../../runners/2.168.0/work/Polymake.jl/Polymake.jl/libcxxwrap-julia-prefix/lib/cmake/JlCxx/../../../include/jlcxx/type_conversion.hpp, line 606.
and a few lines later the crash appears because std::string now has 7048117399172095301
instead of 5163357615
.
After various experiments I can offer you a workaround, to build libcxxwrap-julia yourself with Xcode 11.4 instead of using the binaries:
git clone -b v0.6.6 --depth 1 https://github.com/JuliaInterop/libcxxwrap-julia
cd libcxxwrap-julia
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=$SOMEWHERE/libcxxwrap-julia-install -DCMAKE_BUILD_TYPE=Release ..
VERBOSE=ON cmake --build . --config Release --target install
Then you need to export JLCXX_DIR=$SOMEWHERE/libcxxwrap-julia-install
either in your shell-rc file or in .julia/config/startup.jl
with ENV["JLCXX_DIR"]="<somepath>"
.
FYI: Xcode 11.4.1 has been released three days ago.
First: It seems that this has been fixed in libc++ but it might take some time until this arrives in Xcode, the relevant commit is here: https://github.com/llvm/llvm-project/commit/2464d8135e
Second: I came up with a better, but unfortunately slightly involved, workaround that can be applied to our code. The pull request for Polymake.jl is here, including a detailed description of the issue and workaround: https://github.com/oscar-system/Polymake.jl/pull/270 Github actions seems to be happy with xcode 11.4.
First: It seems that this has been fixed in libc++ but it might take some time until this arrives in Xcode, the relevant commit is here: llvm/llvm-project@2464d81
The commit message states
This commit explicitly enables the assumption of merged typeinfo names
on Apple platform to restore the previous behavior, at least until the
underlying issue has been fixed.
Do you have an idea what they mean by the underlying issue that has to be fixed?
I don't know the details, but I think it is about architectures where the type_info is mostly unique but not always, for example when loading libraries with RTLD_LOCAL
or other special cases.
See the commit message here
https://github.com/llvm/llvm-project/commit/2405bd6898
and this PR where some changes have been reverted:
https://bugs.llvm.org/show_bug.cgi?id=37398
There is also some description of the three type_info variants in the source here: https://github.com/llvm/llvm-project/blob/master/libcxx/include/typeinfo
Thank you very much @benlorenz.
I have bad news :( After updating all packages the problem occurs again.
I have tried the workaround
https://rfourquet.github.io/oscar-website/install/
also with
git clone -b v0.7.0 --depth 1 https://github.com/JuliaInterop/libcxxwrap-julia
but it didn't help.
My Xcode is now 11.4.1.
[c3fe647b] AbstractAlgebra v0.9.1
[9e28174c] BinDeps v1.0.1
[b99e7846] BinaryProvider v0.5.8
[631607c0] CMake v1.2.0
[1f15a43c] CxxWrap v0.10.1
[ffbed154] DocStringExtensions v0.8.1
[e30172f5] Documenter v0.24.10
[c863536a] GAP v0.3.5 [`~/.julia/dev/GAP`]
[50bd374c] HomalgProject v0.2.5 [`~/.julia/dev/HomalgProject`]
[682c06a0] JSON v0.21.0
[472f376f] LoadFlint v0.1.3
[1914dd2f] MacroTools v0.5.5
[2edaba10] Nemo v0.17.5
[69de0a69] Parsers v1.0.2
[bcd08a7b] Singular v0.3.0
[30578b45] URIParser v0.4.1
[3eaa8342] libcxxwrap_julia_jll v0.7.0+0
[2a0f44e3] Base64
[ade2ca70] Dates
[8ba89e20] Distributed
[b77e0a4c] InteractiveUtils
[76f85450] LibGit2
[8f399da3] Libdl
[37e2e46d] LinearAlgebra
[56ddb016] Logging
[d6f4376e] Markdown
[a63ad114] Mmap
[44cfe95a] Pkg
[de0858da] Printf
[3fa0cd96] REPL
[9a3f8284] Random
[ea8e919c] SHA
[9e88b42a] Serialization
[6462fe0b] Sockets
[2f01184e] SparseArrays
[10745b16] Statistics
[8dfed614] Test
[cf7118a7] UUIDs
[4ec0a83e] Unicode
Assertion failed: (has_julia_type<T>()), function value, file /Users/mo/.julia/artifacts/f598f80bcf73532d9dd4a75d657d00235079a949/include/jlcxx/type_conversion.hpp, line 602.
signal (6): Abort trap: 6
in expression starting at /Users/mo/.julia/packages/Singular/yhKrq/src/LibSingular.jl:11
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 26867007 (Pool: 26864972; Big: 2035); GC: 28
ERROR: LoadError: Failed to precompile Singular [bcd08a7b-43d2-5ff7-b6d4-c458787f915c] to /Users/mo/.julia/compiled/v1.4/Singular/9fz0y_Ert4k.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
[3] _require(::Base.PkgId) at ./loading.jl:1029
[4] require(::Base.PkgId) at ./loading.jl:927
[5] require(::Module, ::Symbol) at ./loading.jl:922
[6] include(::Module, ::String) at ./Base.jl:377
[7] top-level scope at none:2
[8] eval at ./boot.jl:331 [inlined]
[9] eval(::Expr) at ./client.jl:449
[10] top-level scope at ./none:3
in expression starting at /Users/mo/.julia/dev/HomalgProject/src/HomalgProject.jl:32
ERROR: LoadError: Failed to precompile HomalgProject [50bd374c-87b5-11e9-015a-2febe71398bd] to /Users/mo/.julia/compiled/v1.4/HomalgProject/z4ehp_Ert4k.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
[3] _require(::Base.PkgId) at ./loading.jl:1029
[4] require(::Base.PkgId) at ./loading.jl:927
[5] require(::Module, ::Symbol) at ./loading.jl:922
[6] include(::String) at ./client.jl:439
[7] top-level scope at none:6
in expression starting at /Users/mo/.julia/dev/HomalgProject/test/runtests.jl:1
This is expected, the workaround is not yet updated for CxxWrap
versions >= 0.10
. In this case the custom installation needs to be specified in an Overrides.toml
file instead of the environment variable.
Please check the libcxxwrap-julia
readme for a short guide: https://github.com/JuliaInterop/libcxxwrap-julia#using-the-compiled-libcxxwrap-julia-in-cxxwrap
Worked. Again, thank you very much.
So, as we use CxxWrap
version >= 0.10
now in the released version of Oscar, the workaround should be updated on the oscar-website, right? (or did I misunderstand something?).
So, as we use
CxxWrap
version>= 0.10
now in the released version of Oscar, the workaround should be updated on the oscar-website, right? (or did I misunderstand something?).
Yes it should. I was about to write an issue which is now obsolete.
Just to say, @rfourquet did update the website. Also, the current Polymake.jl release contains a built-in workaround, so it doesn't need the instructions on our website anymore. Once Singular.jl also receives such a workaround (see issue https://github.com/oscar-system/Singular.jl/issues/229) then we can remove the instructions from our website again.
Ok noted, I will follow that issue and update the website when this is fixed.
@mohamed-barakat I believe this is resolved?
Yes.