RFC: Fix source build(macOS), REPL mode and misc. upgrading oversights

[Gnimuc]

fix #388, @Roger-luo you might want to give it a test ;)

[Roger-luo]

Still get this.

 Building Cxx → `~/.julia/packages/Cxx/2NIIy/deps/build.log`
┌ Error: Error building `Cxx`:
│ [ Info: Building julia binary build
│ ERROR: LoadError: UndefVarError: JULIA_HOME not defined
│ Stacktrace:
│  [1] top-level scope at logging.jl:312
│  [2] top-level scope at /Users/roger/.julia/packages/Cxx/2NIIy/deps/build.jl:43
│  [3] include at ./boot.jl:317 [inlined]
│  [4] include_relative(::Module, ::String) at ./loading.jl:1041
│  [5] include(::Module, ::String) at ./sysimg.jl:29
│  [6] include(::String) at ./client.jl:388
│  [7] top-level scope at none:0
│ in expression starting at /Users/roger/.julia/packages/Cxx/2NIIy/deps/build.jl:42
│ writing path.jl file
│ Tuning for julia installation at /Users/roger/.virtualenvs/julia/bin with sources possibly at /Users/roger/.virtualenvs/julia/bin/../..
└ @ Pkg.Operations ~/Documents/Repos/julia/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1069

[Gnimuc]

@Roger-luo it looks like you were doing a binary build, but this PR is for fixing source build glitches(e.g. REPL mode). For the binary build, please check out this https://github.com/Keno/Cxx.jl/issues/390#issuecomment-443498820. With this PR, just unzip the tarball to the deps folder and run using Cxx.

[Roger-luo]

I see. This means the binary will be provided separately in the future?

[Gnimuc]

yep, just like other BB2 projects.

[Roger-luo]

Seems fine. But segmentfaults when

using Cxx

C++> #include<#iostream>
true

C++> std::cout << "aa";

[Gnimuc]

@Roger-luo are you working on a source-build or binary-build?

This PR should work well for the source-build on MacOS. For binary-build, please take a look at those comments in #390. It looks like @cdsousa is working on a script for better user installation experience: https://github.com/Keno/Cxx.jl/issues/390#issuecomment-459961157

BTW, make sure you've done a cleanup on Cxx.jl's dep folder before trying any of these methods. I suspect the segment faults are triggered by an old installation.

[Roger-luo]

I built Julia from source, with MKL.

[Gnimuc]

debug build or release build?

[Roger-luo]

release build, I'm rebuilding now, deleted deps/build

julia> versioninfo()
Julia Version 1.1.0
Commit 80516ca202 (2019-01-21 21:24 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.2.0)
  CPU: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
  WORD_SIZE: 64
  LIBM: libimf
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
Environment:
  JULIA_EDITOR = atom

[Roger-luo]

I deleted build and rebuilt the whole thing, now I get

julia> using Cxx

C++ > #include<iostream>
true

C++ > std::cout << "aaa" ;

signal (11): Segmentation fault: 11
in expression starting at no file:0
_ZNK4llvm18IntrusiveRefCntPtrIN5clang11FileManagerEEdeEv at /Users/roger/.julia/dev/Cxx/deps/usr/lib/libcxxffi.dylib (unknown line)
_ZNK5clang16CompilerInstance14getFileManagerEv at /Users/roger/.julia/dev/Cxx/deps/usr/lib/libcxxffi.dylib (unknown line)
EnterSourceFile at /Users/roger/.julia/dev/Cxx/deps/usr/lib/libcxxffi.dylib (unknown line)
unknown function (ip: 0x12b1a8fc8)
unknown function (ip: 0x12b1a8f4c)
macro expansion at /Users/roger/.julia/dev/Cxx/src/cxxstr.jl:701 [inlined]
cxxstr_impl at /Users/roger/.julia/dev/Cxx/src/cxxstr.jl:701
unknown function (ip: 0x12b1a8f2c)
jl_fptr_trampoline at /Users/roger/Documents/Repos/julia/src/gf.c:1864
isTopLevelExpression at /Users/roger/.julia/dev/Cxx/src/CxxREPL/replpane.jl:103
#1 at /Users/roger/.julia/dev/Cxx/src/CxxREPL/replpane.jl:166
jl_apply at /Users/roger/Documents/Repos/julia/src/./julia.h:1571 [inlined]
jl_f__apply at /Users/roger/Documents/Repos/julia/src/builtins.c:556
jl_f__apply_latest at /Users/roger/Documents/Repos/julia/src/builtins.c:594
#invokelatest#1 at ./essentials.jl:742 [inlined]
invokelatest at ./essentials.jl:741 [inlined]
do_respond at /Users/roger/Documents/Repos/julia/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:707
unknown function (ip: 0x12b19abd4)
jl_apply at /Users/roger/Documents/Repos/julia/src/./julia.h:1571 [inlined]
jl_f__apply at /Users/roger/Documents/Repos/julia/src/builtins.c:556
jl_f__apply_latest at /Users/roger/Documents/Repos/julia/src/builtins.c:594
#invokelatest#1 at ./essentials.jl:742 [inlined]
invokelatest at ./essentials.jl:741 [inlined]
run_interface at /Users/roger/Documents/Repos/julia/usr/share/julia/stdlib/v1.1/REPL/src/LineEdit.jl:2273
run_frontend at /Users/roger/Documents/Repos/julia/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:1035
run_repl at /Users/roger/Documents/Repos/julia/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:192
#734 at ./client.jl:362
jfptr_#734_6051 at /Users/roger/.virtualenvs/julia/lib/julia/sys.dylib (unknown line)
jl_apply at /Users/roger/Documents/Repos/julia/src/./julia.h:1571 [inlined]
jl_f__apply at /Users/roger/Documents/Repos/julia/src/builtins.c:556
jl_f__apply_latest at /Users/roger/Documents/Repos/julia/src/builtins.c:594
#invokelatest#1 at ./essentials.jl:742 [inlined]
invokelatest at ./essentials.jl:741 [inlined]
run_main_repl at ./client.jl:346
exec_options at ./client.jl:284
_start at ./client.jl:436
true_main at /Users/roger/.virtualenvs/julia/bin/julia (unknown line)
main at /Users/roger/.virtualenvs/julia/bin/julia (unknown line)
Allocations: 11817660 (Pool: 11815841; Big: 1819); GC: 26
[1]    75462 segmentation fault  julia --color=yes

any idea?

[Gnimuc]

OK, I can reproduce the segmentfault as well with a fresh build. It looks like Cxx failed to init file manager (those two symbol demangled to llvm::IntrusiveRefCntPtr<clang::FileManager>::operator*() const and clang::CompilerInstance::getFileManager() const).

This is probably due to some API changes on the LLVM side(Julia1.0 use LLVM 6.0.0, but Julia1.1 use LLVM 6.0.1), which is not compatible with the current implementation of libcxxffi. (I haven't dived into the code, so this is just a wild guess.)

Welp, Cxx.jl source-build is broken again. :( A workaround is to use binary-build if you do wanna use this package. The libcxxffi binary for Julia 1.0 is compatible with Julia 1.1.

UPDATE: all tests can actually pass without any problem, the segmentation fault is REPL-only.

[Gnimuc]

@Roger-luo the real reason is that the newer GC caught an upgrading oversight.

[Roger-luo]

Ok, this is fixed now. Maybe we don't need 0.7 in the CI now? since it's not supported anymore. And you need to use cp -XR on MacOS in the .travis config, MacOS has some extended attributes that you have to ignore.

[Roger-luo]

Could you fix the travis configuration here?

      if [ ! -f $HOME/early_abort ]; then
          julia --project=. -e 'Pkg.test(coverage=true)' || false
      fi

should be

      if [ ! -f $HOME/early_abort ]; then
          julia --project=. -e 'using Pkg; Pkg.test(coverage=true)' || false
      fi

[Roger-luo]

I think this PR should at least pass linux and OS X CI now (tests are passed on my laptop), and maybe someone with access could review this and merge it into master, then tag a new release. (at least to let people try and find out bugs) @Keno

[Gnimuc]

X-ref: https://github.com/Keno/Cxx.jl/issues/373

[Gnimuc]

@Roger-luo see also #406 CI for Linux is green: https://travis-ci.org/Gnimuc/Cxx.jl

[Keno]

I'm unlikely to have much time to spend on this in the near future, but I'm happy to add people to the repo for commit access and do point reviews as necessary. Does that sound like a plan?

[Roger-luo]

Yes! That would be really nice. @Keno I think a lot people is trying to contribute here.

[Keno]

Alright, transferred to JuliaInterop. Have fun.

[Gnimuc]

closed in favor of #406