Make Cxx.jl ready for production environments post Julia 1.0.0

cdsousa commented 6 years ago

@Keno, first let me recognize and praise you for the amazing work you have been doing both here in Cxx.jl and in Julia language itself.

I'm opening this issue to assess if there are and what are the plans for this package for the Julia 1.x timeline.

I think that Cxx.jl is one of the, among so many, Julia strengths. I've been using it as an interactive C++ compiler and as a thin wrapper over C++ libraries with great joy.

I've been trying myself to port Cxx.jl to Julia 1.0 / LVMM 6.0 (fixed build.jl, replace LLVM patches to this version), and was blocked at bootstrap.cpp incompatibilities with new LLVM. Of course, I soon realized the this is something out of my current capabilities (edit: probably out of the capabilities of anyone but Keno :smile:).

I would be glad that this project not only evolves to Julia 1.x but also becomes more maintainable/contributable.

Petr-Hlavenka commented 6 years ago

Another blocker that will need large effort is to get Cxx running on Windows. As I remember, the problem was the absence of gcc/clang/buildtools on an average windows computer. It might be worth to start an discussion on how to tackle this.

programmeroftheeve commented 6 years ago

Given how BinaryBuilder and BinaryProvider are becoming used, it would make sense to package a compiler and stdlib with that, and use that as a default.

Gnimuc commented 6 years ago

It looks like we need to wait https://github.com/staticfloat/LLVMBuilder/issues/34

cdsousa commented 5 years ago

It seems that https://github.com/staticfloat/LLVMBuilder/issues/34 is not enough, since Cxx.jl seems to rely on "internal" and "not-installed" Clang headers.

See for example https://github.com/Keno/Cxx.jl/blob/master/src/bootstrap.cpp#L96, the CodeGen/CodeGenModule.h header file lives inside clang-6.0.0/lib/CodeGen source folder rather than inside clang-6.0.0/include/clang/CodeGen. The headers in that folder do not get installed.

Besides that, there are a couple #define private public (e.g., https://github.com/Keno/Cxx.jl/blob/master/src/bootstrap.cpp#L58) that makes me beleive that Cxx.jl in fact needs a lot of non-API Clang functionalities...

oschulz commented 5 years ago

Is there a chance that we can make Cxx work on Julia v1.0 until November, if only on Linux? Would be relevant to our current project.

Gnimuc commented 5 years ago

making progress, I'm also investigating how to cross-compile windows binaries.

cdsousa commented 5 years ago

I'm also trying using LLVMBuilder, Clang sources (needed for non-API headers), and BinDeps here: https://github.com/cdsousa/Cxx.jl/blob/binaryBuilder/deps/build.jl So far it compiles libcxxffi.so, but it is very rough and should be seen more as a proof of concept.

Gnimuc commented 5 years ago

I uploaded libcxxffi binaries for Linux and macOS here: https://github.com/Gnimuc/LLVMBuilder/releases/tag/0.0.0-2

~~I'm looking for a way to replace dlfcn.h for windows.~~ Code cross-compiled on MinGW, but the dll cannot be dlopened even the dependency list in dependency walker looks OK.

I guess I did something wrong when linking those libs. Linux & macOS binaries cannot work in the REPL mode too.

cdsousa commented 5 years ago

I uploaded libcxxffi binaries for Linux and macOS here: https://github.com/Gnimuc/LLVMBuilder/releases/tag/0.0.0-2

@Gnimuc, very nice, thanks! I tried it in Linux x86_64 and it worker very well. I have put my build.jl (modified from LLVMBuilder) here: https://github.com/cdsousa/Cxx.jl/tree/fullBinaryBuilder/deps

Gnimuc commented 5 years ago

hi, @cdsousa, did you get cxx REPL work? I get the following errors even when building from source.

C++ > int a{2};
ERROR: UndefVarError: parser not defined
Stacktrace:
 [1] isTopLevelExpression(::Cxx.CxxCore.CxxInstance{1}, ::String) at /Users/gnimuc/.julia/dev/Cxx/src/CxxREPL/replpane.jl:102
 [2] (::getfield(Cxx.CxxREPL, Symbol("##1#2")){Cxx.CxxCore.CxxInstance{1}})(::String) at /Users/gnimuc/.julia/dev/Cxx/src/CxxREPL/replpane.jl:165
 [3] #invokelatest#1 at ./essentials.jl:697 [inlined]
 [4] invokelatest at ./essentials.jl:696 [inlined]
 [5] (::getfield(REPL, Symbol("#do_respond#38")){Bool,getfield(Cxx.CxxREPL, Symbol("##1#2")){Cxx.CxxCore.CxxInstance{1}},REPL.LineEditREPL,REPL.LineEdit.Prompt})(::Any, ::Any, ::Any) at /Users/gnimuc/Codes/julia/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:707
 [6] #invokelatest#1 at ./essentials.jl:697 [inlined]
 [7] invokelatest at ./essentials.jl:696 [inlined]
 [8] run_interface(::REPL.Terminals.TextTerminal, ::REPL.LineEdit.ModalInterface, ::REPL.LineEdit.MIState) at /Users/gnimuc/Codes/julia/usr/share/julia/stdlib/v1.1/REPL/src/LineEdit.jl:2273
 [9] run_frontend(::REPL.LineEditREPL, ::REPL.REPLBackendRef) at /Users/gnimuc/Codes/julia/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:1035
 [10] run_repl(::REPL.AbstractREPL, ::Any) at /Users/gnimuc/Codes/julia/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:192
 [11] (::getfield(Base, Symbol("##726#728")){Bool,Bool,Bool,Bool})(::Module) at ./client.jl:362
 [12] #invokelatest#1 at ./essentials.jl:697 [inlined]
 [13] invokelatest at ./essentials.jl:696 [inlined]
 [14] run_main_repl(::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at ./client.jl:346
 [15] exec_options(::Base.JLOptions) at ./client.jl:284
 [16] _start() at ./client.jl:436

cdsousa commented 5 years ago

No, I haven't either.

Gnimuc commented 5 years ago

@cdsousa REPL mode is fixed in #400.

cdsousa commented 5 years ago

A POC version using BinDeps (and LLVMBuilder and Clang source) is now working in Julia 1.0.3 in Ubuntu 18.04 / x86_64 (https://github.com/cdsousa/Cxx.jl/tree/bindeps):

import Pkg; Pkg.clone("https://github.com/cdsousa/Cxx.jl.git", "bindeps"); Pkg.build("Cxx")
using Cxx
cxx"#include <iostream>"
icxx"std::cout << \"Hello world!\" << std::endl;"

I have a preference for a solution like this, as it does not require a new binaryBuilder for Cxx.

ahumenberger commented 5 years ago

A POC version using BinDeps (and LLVMBuilder and Clang source) is now working in Julia 1.0.3 in Ubuntu 18.04 / x86_64 (https://github.com/cdsousa/Cxx.jl/tree/bindeps):
import Pkg; Pkg.clone("https://github.com/cdsousa/Cxx.jl.git", "bindeps"); Pkg.build("Cxx")
using Cxx
cxx"#include <iostream>"
icxx"std::cout << \"Hello world!\" << std::endl;"
I have a preference for a solution like this, as it does not require a new binaryBuilder for Cxx.

When I'm trying that on my macOS Mojave and Julia 1.0.3 I'm getting the following:

ERROR: LoadError: MethodError: no method matching VersionNumber(::VersionNumber)

It seems like the wrong branch is checked out. However if I'm using add https://github.com/cdsousa/Cxx.jl.git#bindeps I'm getting:

┌ Error: Error building Cxx: 
│ : CommandLine Error: Option 'asm-instrumentation' registered more than once!
│ LLVM ERROR: inconsistency in registered CommandLine options
└ @ Pkg.Operations /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1097

Does anyone know the cause of this error?

cdsousa commented 5 years ago

Does anyone know the cause of this error?

Yeah, the POC is simply not working in macOS. I also use macOS from time to time, but I had no time to make it work there.

cdsousa commented 5 years ago

Hi @Gnimuc and others, I've been trying to use Cxx in macOS in a Julia 1.1 binary installation (LLVM 6.0.1), and so far I've been able to successfully build (`Pkg.build) Cxx with:

your "binaryBuilder" version (https://github.com/Gnimuc/LLVMBuilder/releases/tag/0.0.0-2 / https://github.com/cdsousa/Cxx.jl/tree/fullBinaryBuilder)
my "binDeps" version (https://github.com/cdsousa/Cxx.jl/tree/bindeps_j1.1)
standard way (corrected by you and me) by building LLVM and Clang (https://github.com/cdsousa/Cxx.jl/tree/patch-mac)

The problem is that when doing using Cxx I get several compilation errors, sometimes due to a missing "string.h" header, but more frequently this one:

In file included from /Cxx.cpp:1:
In file included from /Cxx.h:1:
In file included from /Users/csousa/.julia/dev/Cxx/src/CxxREPL/replpane.jl:70:
In file included from /Users/csousa/.julia/dev/Cxx/src/CxxREPL/../../deps/src/clang-6.0.1/include/clang/Parse/Parser.h:17:
In file included from /Users/csousa/.julia/dev/Cxx/src/CxxREPL/../../deps/src/clang-6.0.1/include/clang/AST/Availability.h:17:
In file included from /Users/csousa/.julia/dev/Cxx/src/CxxREPL/../../deps/src/clang-6.0.1/include/clang/Basic/SourceLocation.h:23:
/Library/Developer/CommandLineTools/usr/include/c++/v1/string:652:38: error: no type named 'const_pointer' in 'std::__1::allocator_traits<std::__1::allocator<char> >'
    typedef typename __alloc_traits::const_pointer       const_pointer;
            ~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~

Although this is happening in Julia 1.1 with LLVM 6.0.1, I did several tries with Julia 1.0.3 (LLVM 6.0.0) getting the same errors.

Any idea?

(Edit: @ahumenberger, I also got your error but worked around it in new current binDeps and binDeps_1.1 branches. Can you try again please?)

Gnimuc commented 5 years ago

I haven't try LLVM 6.0.1 yet, because it turns out the LLVM 6.0.0 is upward compatible. I just tested the binary build version of the package again and here are some reports:

Source Build

Should be OK with the patch-mac branch on both Linux and macOS.

Binary Build

Julia 1.1 + BB2(LLVM 6.0.0) on macOS

julia> using Cxx
[ Info: Recompiling stale cache file /Users/gnimuc/.julia/compiled/v1.1/Cxx/ESGkI.ji for Cxx [a0b5b9ef-44b7-5148-a2d1-f6db19f3c3d2]

(v1.1) pkg> test Cxx
   Testing Cxx
 Resolving package versions...
    Status `/var/folders/ft/nd_bm3z52152069y78vb71280000gn/T/tmp9Mv8Vg/Manifest.toml`
 ...
Test Summary:                   | Pass  Total
import without using Cxx (#316) |    2      2
foo
...
10
Test Summary: | Pass  Total
StdString     |    3      3
Test Summary: | Pass  Total
StdVector     |   70     70
e = (class std::length_error &) {
}

Test Summary: | Pass  Total
Exceptions    |    2      2
   Testing Cxx tests passed

Julia 1.0 + BB2(LLVM 6.0.0) on macOS

julia> using Cxx
[ Info: Recompiling stale cache file /Users/gnimuc/.julia/compiled/v1.0/Cxx/ESGkI.ji for Cxx [a0b5b9ef-44b7-5148-a2d1-f6db19f3c3d2]

(v1.0) pkg> test Cxx
   Testing Cxx
 Resolving package versions...
    Status `/var/folders/ft/nd_bm3z52152069y78vb71280000gn/T/tmpuToUtK/Manifest.toml`
...
Test Summary:                   | Pass  Total
import without using Cxx (#316) |    2      2
foo
...
10
Test Summary: | Pass  Total
StdString     |    3      3
StdVector copy(to)! and convert: Error During Test at /Users/gnimuc/.julia/dev/Cxx/test/std.jl:179
  Test threw exception
  Expression: begin
    dest = zeros(Int32, 7)
    copy!(dest, cxx_int_v)
    dest == jl_int_v
end
  MethodError: no method matching copy!(::Array{Int32,1}, ::Cxx.CxxStd.WrappedCppPrimArray{Int32})
  Closest candidates are:
    copy!(!Matched::Union{CppRef{CppTemplate{CppBaseType{Symbol("std::__1::vector")},Tuple{T,CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::allocator")},Tuple{T}},(false, false, false)}}},(false, false, false)}, CppValue{CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::vector")},Tuple{T,CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::allocator")},Tuple{T}},(false, false, false)}}},(false, false, false)},N} where N} where T, ::Any) at /Users/gnimuc/.julia/dev/Cxx/src/std.jl:114
    copy!(::AbstractArray, !Matched::Union{CppRef{CppTemplate{CppBaseType{Symbol("std::__1::vector")},Tuple{T,CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::allocator")},Tuple{T}},(false, false, false)}}},(false, false, false)}, CppValue{CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::vector")},Tuple{T,CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::allocator")},Tuple{T}},(false, false, false)}}},(false, false, false)},N} where N} where T) at /Users/gnimuc/.julia/dev/Cxx/src/std.jl:115
    copy!(!Matched::Union{WrappedCppBoolVector, WrappedCppObjArray, WrappedCppPrimArray}, ::AbstractArray{T,1} where T) at /Users/gnimuc/.julia/dev/Cxx/src/std.jl:204
    ...
  Stacktrace:
   [1] copy!(::Array{Int32,1}, ::CppValue{CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::vector")},Tuple{Int32,CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::allocator")},Tuple{Int32}},(false, false, false)}}},(false, false, false)},24}) at /Users/gnimuc/.julia/dev/Cxx/src/std.jl:115
   [2] macro expansion at /Users/gnimuc/.julia/dev/Cxx/test/std.jl:179 [inlined]
   [3] macro expansion at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [4] macro expansion at /Users/gnimuc/.julia/dev/Cxx/test/std.jl:179 [inlined]
   [5] macro expansion at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [6] top-level scope at /Users/gnimuc/.julia/dev/Cxx/test/std.jl:16
StdVector copy(to)! and convert: Error During Test at /Users/gnimuc/.julia/dev/Cxx/test/std.jl:206
  Test threw exception
  Expression: begin
    dest = Vector{String}(undef, 3)
    copy!(dest, cxx_str_v)
    dest == jl_str_v
end
  MethodError: no method matching copy!(::Array{String,1}, ::Cxx.CxxStd.WrappedCppObjArray{CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::basic_string")},Tuple{UInt8,CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::char_traits")},Tuple{UInt8}},(false, false, false)},CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::allocator")},Tuple{UInt8}},(false, false, false)}}},(false, false, false)},(false, false, false)})
  Closest candidates are:
    copy!(!Matched::Union{CppRef{CppTemplate{CppBaseType{Symbol("std::__1::vector")},Tuple{T,CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::allocator")},Tuple{T}},(false, false, false)}}},(false, false, false)}, CppValue{CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::vector")},Tuple{T,CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::allocator")},Tuple{T}},(false, false, false)}}},(false, false, false)},N} where N} where T, ::Any) at /Users/gnimuc/.julia/dev/Cxx/src/std.jl:114
    copy!(::AbstractArray, !Matched::Union{CppRef{CppTemplate{CppBaseType{Symbol("std::__1::vector")},Tuple{T,CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::allocator")},Tuple{T}},(false, false, false)}}},(false, false, false)}, CppValue{CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::vector")},Tuple{T,CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::allocator")},Tuple{T}},(false, false, false)}}},(false, false, false)},N} where N} where T) at /Users/gnimuc/.julia/dev/Cxx/src/std.jl:115
    copy!(!Matched::Union{WrappedCppBoolVector, WrappedCppObjArray, WrappedCppPrimArray}, ::AbstractArray{T,1} where T) at /Users/gnimuc/.julia/dev/Cxx/src/std.jl:204
    ...
  Stacktrace:
   [1] copy!(::Array{String,1}, ::CppValue{CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::vector")},Tuple{CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::basic_string")},Tuple{UInt8,CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::char_traits")},Tuple{UInt8}},(false, false, false)},CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::allocator")},Tuple{UInt8}},(false, false, false)}}},(false, false, false)},CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::allocator")},Tuple{CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::basic_string")},Tuple{UInt8,CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::char_traits")},Tuple{UInt8}},(false, false, false)},CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::allocator")},Tuple{UInt8}},(false, false, false)}}},(false, false, false)}}},(false, false, false)}}},(false, false, false)},24}) at /Users/gnimuc/.julia/dev/Cxx/src/std.jl:115
   [2] macro expansion at /Users/gnimuc/.julia/dev/Cxx/test/std.jl:206 [inlined]
   [3] macro expansion at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [4] macro expansion at /Users/gnimuc/.julia/dev/Cxx/test/std.jl:179 [inlined]
   [5] macro expansion at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [6] top-level scope at /Users/gnimuc/.julia/dev/Cxx/test/std.jl:16
StdVector copy(to)! and convert: Error During Test at /Users/gnimuc/.julia/dev/Cxx/test/std.jl:221
  Test threw exception
  Expression: begin
    dest = zeros(Bool, 4)
    copy!(dest, cxx_bool_v)
    dest == jl_bool_v
end
  MethodError: no method matching copy!(::Array{Bool,1}, ::Cxx.CxxStd.WrappedCppBoolVector)
  Closest candidates are:
    copy!(!Matched::Union{CppRef{CppTemplate{CppBaseType{Symbol("std::__1::vector")},Tuple{T,CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::allocator")},Tuple{T}},(false, false, false)}}},(false, false, false)}, CppValue{CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::vector")},Tuple{T,CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::allocator")},Tuple{T}},(false, false, false)}}},(false, false, false)},N} where N} where T, ::Any) at /Users/gnimuc/.julia/dev/Cxx/src/std.jl:114
    copy!(::AbstractArray, !Matched::Union{CppRef{CppTemplate{CppBaseType{Symbol("std::__1::vector")},Tuple{T,CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::allocator")},Tuple{T}},(false, false, false)}}},(false, false, false)}, CppValue{CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::vector")},Tuple{T,CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::allocator")},Tuple{T}},(false, false, false)}}},(false, false, false)},N} where N} where T) at /Users/gnimuc/.julia/dev/Cxx/src/std.jl:115
    copy!(!Matched::Union{WrappedCppBoolVector, WrappedCppObjArray, WrappedCppPrimArray}, ::AbstractArray{T,1} where T) at /Users/gnimuc/.julia/dev/Cxx/src/std.jl:204
    ...
  Stacktrace:
   [1] copy!(::Array{Bool,1}, ::CppValue{CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::vector")},Tuple{Bool,CxxQualType{CppTemplate{CppBaseType{Symbol("std::__1::allocator")},Tuple{Bool}},(false, false, false)}}},(false, false, false)},24}) at /Users/gnimuc/.julia/dev/Cxx/src/std.jl:115
   [2] macro expansion at /Users/gnimuc/.julia/dev/Cxx/test/std.jl:221 [inlined]
   [3] macro expansion at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [4] macro expansion at /Users/gnimuc/.julia/dev/Cxx/test/std.jl:179 [inlined]
   [5] macro expansion at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [6] top-level scope at /Users/gnimuc/.julia/dev/Cxx/test/std.jl:16
Test Summary:                               | Pass  Error  Total
StdVector                                   |   67      3     70
  StdVector basic methods                   |   19            19
  StdVector iteration                       |    1             1
  StdVector getindex and setindex!          |   11            11
  StdVector wrappers basic methods          |   16            16
  StdVector wrappers getindex and setindex! |   11            11
  StdVector copy(to)! and convert           |    9      3     12
ERROR: LoadError: LoadError: Some tests did not pass: 67 passed, 0 failed, 3 errored, 0 broken.
in expression starting at /Users/gnimuc/.julia/dev/Cxx/test/std.jl:15
in expression starting at /Users/gnimuc/.julia/dev/Cxx/test/runtests.jl:20
ERROR: Package Cxx errored during testing

Julia 1.0 + BB2(LLVM 6.0.0) on Linux

julia> using Cxx

(v1.0) pkg> test Cxx
   Testing Cxx
 Resolving package versions...
    Status `/tmp/tmpMSBkLT/Manifest.toml`
  ...
Test Summary:                   | Pass  Total
import without using Cxx (#316) |    2      2
foo
...
10
Test Summary: | Pass  Total
StdString     |    3      3
StdVector copy(to)! and convert: Error During Test at /home/parallels/.julia/dev/Cxx/test/std.jl:179
  Test threw exception
  Expression: begin
    dest = zeros(Int32, 7)
    copy!(dest, cxx_int_v)
    dest == jl_int_v
end
  MethodError: no method matching copy!(::Array{Int32,1}, ::Cxx.CxxStd.WrappedCppPrimArray{Int32})
  Closest candidates are:
    copy!(!Matched::Union{CppRef{CppTemplate{CppBaseType{Symbol("std::vector")},Tuple{T,CxxQualType{CppTemplate{CppBaseType{Symbol("std::allocator")},Tuple{T}},(false, false, false)}}},(false, false, false)}, CppValue{CxxQualType{CppTemplate{CppBaseType{Symbol("std::vector")},Tuple{T,CxxQualType{CppTemplate{CppBaseType{Symbol("std::allocator")},Tuple{T}},(false, false, false)}}},(false, false, false)},N} where N} where T, ::Any) at /home/parallels/.julia/dev/Cxx/src/std.jl:114
    copy!(::AbstractArray, !Matched::Union{CppRef{CppTemplate{CppBaseType{Symbol("std::vector")},Tuple{T,CxxQualType{CppTemplate{CppBaseType{Symbol("std::allocator")},Tuple{T}},(false, false, false)}}},(false, false, false)}, CppValue{CxxQualType{CppTemplate{CppBaseType{Symbol("std::vector")},Tuple{T,CxxQualType{CppTemplate{CppBaseType{Symbol("std::allocator")},Tuple{T}},(false, false, false)}}},(false, false, false)},N} where N} where T) at /home/parallels/.julia/dev/Cxx/src/std.jl:115
    copy!(!Matched::Union{WrappedCppBoolVector, WrappedCppObjArray, WrappedCppPrimArray}, ::AbstractArray{T,1} where T) at /home/parallels/.julia/dev/Cxx/src/std.jl:204
    ...
  Stacktrace:
   [1] copy!(::Array{Int32,1}, ::CppValue{CxxQualType{CppTemplate{CppBaseType{Symbol("std::vector")},Tuple{Int32,CxxQualType{CppTemplate{CppBaseType{Symbol("std::allocator")},Tuple{Int32}},(false, false, false)}}},(false, false, false)},24}) at /home/parallels/.julia/dev/Cxx/src/std.jl:115
   [2] macro expansion at /home/parallels/.julia/dev/Cxx/test/std.jl:37 [inlined]
   [3] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [4] macro expansion at /home/parallels/.julia/dev/Cxx/test/std.jl:26 [inlined]
   [5] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [6] top-level scope at /home/parallels/.julia/dev/Cxx/test/std.jl:16
In file included from /Cxx.cpp:1:
:3:35: error: expected unqualified-id
            auto &r = __juliavar1.;
                                  ^
StdVector copy(to)! and convert: Error During Test at /home/parallels/.julia/dev/Cxx/test/std.jl:178
  Got exception outside of a @test
  UndefVarError: contains not defined
  Stacktrace:
   [1] #s71#1 at /home/parallels/.julia/dev/Cxx/src/show.jl:13 [inlined]
   [2] #s71#1(::Any, ::Any, ::Any) at ./none:0
   [3] (::Core.GeneratedFunctionStub)(::Any, ::Vararg{Any,N} where N) at ./boot.jl:506
   [4] macro expansion at /home/parallels/.julia/dev/Cxx/src/show.jl:7 [inlined]
   [5] show(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::CppRef{CppTemplate{CppBaseType{Symbol("std::__cxx11::basic_string")},Tuple{UInt8,CxxQualType{CppTemplate{CppBaseType{Symbol("std::char_traits")},Tuple{UInt8}},(false, false, false)},CxxQualType{CppTemplate{CppBaseType{Symbol("std::allocator")},Tuple{UInt8}},(false, false, false)}}},(false, false, false)}) at /home/parallels/.julia/dev/Cxx/src/show.jl:9
   [6] show_delim_array(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Cxx.CxxStd.WrappedCppObjArray{CxxQualType{CppTemplate{CppBaseType{Symbol("std::__cxx11::basic_string")},Tuple{UInt8,CxxQualType{CppTemplate{CppBaseType{Symbol("std::char_traits")},Tuple{UInt8}},(false, false, false)},CxxQualType{CppTemplate{CppBaseType{Symbol("std::allocator")},Tuple{UInt8}},(false, false, false)}}},(false, false, false)},(false, false, false)}, ::Char, ::String, ::Char, ::Bool, ::Int64, ::Int64) at ./show.jl:663
   [7] show_delim_array at ./show.jl:649 [inlined]
   [8] show_vector(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Cxx.CxxStd.WrappedCppObjArray{CxxQualType{CppTemplate{CppBaseType{Symbol("std::__cxx11::basic_string")},Tuple{UInt8,CxxQualType{CppTemplate{CppBaseType{Symbol("std::char_traits")},Tuple{UInt8}},(false, false, false)},CxxQualType{CppTemplate{CppBaseType{Symbol("std::allocator")},Tuple{UInt8}},(false, false, false)}}},(false, false, false)},(false, false, false)}, ::Char, ::Char) at ./arrayshow.jl:442
   [9] show_vector at ./arrayshow.jl:432 [inlined]
   [10] show at ./arrayshow.jl:418 [inlined]
   [11] show_delim_array(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Tuple{Array{String,1},Cxx.CxxStd.WrappedCppObjArray{CxxQualType{CppTemplate{CppBaseType{Symbol("std::__cxx11::basic_string")},Tuple{UInt8,CxxQualType{CppTemplate{CppBaseType{Symbol("std::char_traits")},Tuple{UInt8}},(false, false, false)},CxxQualType{CppTemplate{CppBaseType{Symbol("std::allocator")},Tuple{UInt8}},(false, false, false)}}},(false, false, false)},(false, false, false)}}, ::Char, ::Char, ::Char, ::Bool, ::Int64, ::Int64) at ./show.jl:695
   [12] show_delim_array at ./show.jl:680 [inlined]
   [13] show(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Tuple{Array{String,1},Cxx.CxxStd.WrappedCppObjArray{CxxQualType{CppTemplate{CppBaseType{Symbol("std::__cxx11::basic_string")},Tuple{UInt8,CxxQualType{CppTemplate{CppBaseType{Symbol("std::char_traits")},Tuple{UInt8}},(false, false, false)},CxxQualType{CppTemplate{CppBaseType{Symbol("std::allocator")},Tuple{UInt8}},(false, false, false)}}},(false, false, false)},(false, false, false)}}) at ./show.jl:714
   [14] show_default(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Any) at ./show.jl:332
   [15] show(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Any) at ./show.jl:315
   [16] #sprint#325(::Pair{Symbol,Bool}, ::Int64, ::Function, ::Function, ::MethodError) at ./strings/io.jl:99
   [17] #sprint at ./none:0 [inlined]
   [18] Test.Error(::Symbol, ::Expr, ::MethodError, ::Array{Union{Ptr{Nothing}, InterpreterIP},1}, ::LineNumberNode) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:144
   [19] do_test(::Test.Threw, ::Expr) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:494
   [20] macro expansion at /home/parallels/.julia/dev/Cxx/test/std.jl:46 [inlined]
   [21] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [22] macro expansion at /home/parallels/.julia/dev/Cxx/test/std.jl:26 [inlined]
   [23] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [24] top-level scope at /home/parallels/.julia/dev/Cxx/test/std.jl:16
   [25] include at ./boot.jl:317 [inlined]
   [26] include_relative(::Module, ::String) at ./loading.jl:1038
   [27] include(::Module, ::String) at ./sysimg.jl:29
   [28] include(::String) at ./client.jl:388
   [29] top-level scope at none:0
   [30] include at ./boot.jl:317 [inlined]
   [31] include_relative(::Module, ::String) at ./loading.jl:1038
   [32] include(::Module, ::String) at ./sysimg.jl:29
   [33] include(::String) at ./client.jl:388
   [34] top-level scope at none:0
   [35] eval(::Module, ::Any) at ./boot.jl:319
   [36] macro expansion at ./logging.jl:317 [inlined]
   [37] exec_options(::Base.JLOptions) at ./client.jl:219
   [38] _start() at ./client.jl:421
Test Summary:                               | Pass  Error  Total
StdVector                                   |   62      2     64
  StdVector basic methods                   |   19            19
  StdVector iteration                       |    1             1
  StdVector getindex and setindex!          |   11            11
  StdVector wrappers basic methods          |   16            16
  StdVector wrappers getindex and setindex! |   11            11
  StdVector copy(to)! and convert           |    4      2      6
ERROR: LoadError: LoadError: Some tests did not pass: 62 passed, 0 failed, 2 errored, 0 broken.
in expression starting at /home/parallels/.julia/dev/Cxx/test/std.jl:15
in expression starting at /home/parallels/.julia/dev/Cxx/test/runtests.jl:20
ERROR: Package Cxx errored during testing

Julia 1.1 + BB2(LLVM 6.0.0) on Linux

julia> using Cxx
[ Info: Precompiling Cxx [a0b5b9ef-44b7-5148-a2d1-f6db19f3c3d2]

(v1.1) pkg> test Cxx
   Testing Cxx
 Resolving package versions...
    Status `/tmp/tmpxbfuac/Manifest.toml`
...
Test Summary:                   | Pass  Total
import without using Cxx (#316) |    2      2
foo
...
10
Test Summary: | Pass  Total
StdString     |    3      3
Test Summary: | Pass  Total
StdVector     |   70     70
e = (class std::length_error &) {
}

Test Summary: | Pass  Total
Exceptions    |    2      2
   Testing Cxx tests passed

Gnimuc commented 5 years ago

Although this is happening in Julia 1.1 with LLVM 6.0.1, I did several tries with Julia 1.0.3 (LLVM 6.0.0) getting the same errors.

Not sure, but if you dev the package both on Julia 1.0 and 1.1, the environment might be "polluted".

BTW, since Cxx.jl automatically choose LLVM version simply based on Julia's LLVM version, you may need to hard-code replpane.jl line 52 and initialization.jl line 371 to "6.0.0" if you wanna use Julia 1.1 + LLVM 6.0.0.

cdsousa commented 5 years ago

@Gnimuc, @ahumenberger and others, I've finally managed to get Cxx working in macOS in a Julia 1.1 binary installation (LLVM 6.0.1) with my "binDeps" version lastest commit https://github.com/cdsousa/Cxx.jl/tree/bindeps_j1.1 (there was some bug related to macOS include directories).

One issue still around is that the Pkg.build("Cxx") still ends failling with a CommandLine Error: Option 'executor' registered more than once! │ LLVM ERROR: inconsistency in registered CommandLine options, although that doesn't compromise the creation of the necessary libcxxffi library.

Gnimuc commented 5 years ago

FYI, BB2 support for Julia 1.1 is also working in progress: #406

cdsousa commented 5 years ago

@Gnimuc I'm pretty happy with the current state (I use Linux and MacOS but not Windows)... I agree on closing this issue when a new version is tagged so that pkg> add Cxx just works.

cdsousa commented 5 years ago

pkg> add Cxx just works!

Let's make further "production environment readiness" discussions in separate, more actionable, issues.

JuliaInterop / Cxx.jl