Windows issues - Githubissues

Petr-Hlavenka commented 5 years ago

Thank you @Gnimuc for tackling the llvm build via BB. Installs fine on windows and as mentioned in the readme, the windows experience is rough now.

example3:

julia> @cxx printme(pointer("John"))
LLVM ERROR: Program used external function '__gxx_personality_v0' which could not be resolved!

this looks like problem of not linking against -lstdc++ or similar as discussed e.g. here: https://stackoverflow.com/questions/329059/what-is-gxx-personality-v0-for Is it hard to play with the switches on the BB? Being windows-only the linux/gcc way of app building is a black magic to me.

Gnimuc commented 5 years ago

I encountered that before but had no idea how to fix it. The Windows binary is cross-compiled in BB2's mingw shards. Here is my experimental repo: https://github.com/Gnimuc/CxxBuilder the build procedures are almost the same for all platforms.

MaximeBouton commented 5 years ago

Hi, I tried to add the following line in the CxxBuilder script provided by @Gnimuc :

if [[ ${target} == *mingw32* ]] && [[ ${nbits} == 64 ]]; then
   CMAKE_FLAGS="${CMAKE_FLAGS} -DCMAKE_CXX_FLAGS=-fno-exceptions"
fi

but the build does not succeed locally, I am getting the following error (regardless of my change to build_tarballs.jl):

[19:14:40]  ---> cd $WORKSPACE/srcdir/
[19:14:40]  ---> mkdir LLVMBinary
[19:14:40]  ---> tar -C LLVMBinary --strip-components=1 -xf *LLVM*.tar.gz
[19:14:49]  ---> cd $WORKSPACE/srcdir/
[19:14:49]  ---> mkdir build
[19:14:49]  ---> cd build
[19:14:49]  ---> CMAKE_FLAGS="-DCMAKE_INSTALL_PREFIX=$prefix -DCMAKE_TOOLCHAIN_FILE=/opt/$target/$target.toolchain"
[19:14:49]  ---> CMAKE_FLAGS="${CMAKE_FLAGS} -DJULIA_SOURCE_PREFIX=$WORKSPACE/srcdir/julia"
[19:14:49]  ---> CMAKE_FLAGS="${CMAKE_FLAGS} -DJULIA_BINARY_PREFIX=$WORKSPACE/srcdir/juliabin"
[19:14:49]  ---> CMAKE_FLAGS="${CMAKE_FLAGS} -DLLVMBUILDER_PREFIX=$WORKSPACE/srcdir/LLVMBinary"
[19:14:49]  ---> [[ ${target} == *mingw32* ]]
[19:14:49]  ---> [[ ${nbits} == 64 ]]
[19:14:49]  ---> CMAKE_FLAGS="${CMAKE_FLAGS} -DCMAKE_CXX_FLAGS=-fno-exceptions"
[19:14:49]  ---> cmake .. ${CMAKE_FLAGS}
[19:14:49] -- The C compiler identification is GNU 7.1.0
[19:14:49] -- The CXX compiler identification is GNU 7.1.0
[19:14:49] -- Check for working C compiler: /opt/x86_64-w64-mingw32/bin/x86_64-w64-mingw32-gcc
[19:14:49] -- Check for working C compiler: /opt/x86_64-w64-mingw32/bin/x86_64-w64-mingw32-gcc -- works
[19:14:49] -- Detecting C compiler ABI info
[19:14:50] -- Detecting C compiler ABI info - done
[19:14:50] -- Detecting C compile features
[19:14:50] -- Detecting C compile features - done
[19:14:50] -- Check for working CXX compiler: /opt/x86_64-w64-mingw32/bin/x86_64-w64-mingw32-g++
[19:14:50] -- Check for working CXX compiler: /opt/x86_64-w64-mingw32/bin/x86_64-w64-mingw32-g++ -- works
[19:14:50] -- Detecting CXX compiler ABI info
[19:14:50] -- Detecting CXX compiler ABI info - done
[19:14:50] -- Detecting CXX compile features
[19:14:51] -- Detecting CXX compile features - done
[19:14:51] -- JULIA_SOURCE_INCLUDE_DIRS: /workspace/srcdir/julia/src/support
[19:14:51] -- JULIA_INCLUDE_DIRS: /workspace/srcdir/juliabin/include/julia
[19:14:51] -- Julia_LIB_DIRS: /workspace/srcdir/juliabin/bin
[19:14:51] -- Julia_LIB_DIRS: /workspace/srcdir/juliabin/lib
[19:14:51] -- Julia_LIB_DIRS: /workspace/srcdir/juliabin/lib/julia
[19:14:51] -- LLVM_VERSION: 6.0.1
[19:14:51] -- use default CLANG_SOURCE_PREFIX: /workspace/srcdir/llvm-6.0.1.src/tools/clang
[19:14:51] -- CLANG_SOURCE_INCLUDE_DIRS: /workspace/srcdir/llvm-6.0.1.src/tools/clang/include
[19:14:51] -- CLANG_SOURCE_INCLUDE_DIRS: /workspace/srcdir/llvm-6.0.1.src/tools/clang/lib
[19:14:51] -- CLANG_INCLUDE_DIRS: /workspace/srcdir/LLVMBinary/include
[19:14:51] -- CLANG_LIB_DIRS: /workspace/srcdir/LLVMBinary/lib
[19:14:51] -- JULIA_LIBS: JULIA_LIBS-NOTFOUND
[19:14:51] -- LLVM_LIBS: LLVM_LIBS-NOTFOUND
[19:14:51] CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
[19:14:51] Please set them or make sure they are set and tested correctly in the CMake files:
[19:14:51] JULIA_LIBS
[19:14:51]     linked by target "cxxffi" in directory /workspace/srcdir
[19:14:51] LLVM_LIBS
[19:14:51]     linked by target "cxxffi" in directory /workspace/srcdir
[19:14:51] 
[19:14:51] -- Configuring incomplete, errors occurred!
[19:14:51] See also "/workspace/srcdir/build/CMakeFiles/CMakeOutput.log".
[19:14:51]  ---> cmake .. ${CMAKE_FLAGS}
[19:14:51]  ---> save_srcdir
[19:14:51] Saving srcdir due to previous error...
[19:15:04]  ---> cmake .. ${CMAKE_FLAGS}

Do you have any idea how to fix this error? Where should JULIA_LIBS and LLVM_LIBS be defined?

Is the error above due to libcxx or llvm?

edit: I was not using the right version of llvm and julia I uploaded the binary compiled with the new cmake file here: https://github.com/MaximeBouton/CxxBuilder/releases/tag/v0.0.4-1 Building Cxx works, but when I call using Cxx, I am getting the following:

julia> using Cxx
[ Info: Precompiling Cxx [a0b5b9ef-44b7-5148-a2d1-f6db19f3c3d2]
A s s e r t i o n   f a i l e d !

 P r o g r a m :   C : \ P r o g r a m   F i l e s \ J u l i a - 1 . 1 . 0 \ b i n \ j u l i a . e x e
 F i l e :   / w o r k s p a c e / s r c d i r / b o o t s t r a p . c p p ,   L i n e   1 5 2 3

 E x p r e s s i o n :   f _ j u l i a _ t y p e _ t o _ l l v m

signal (22): SIGABRT
in expression starting at C:\Users\Maxime\.julia\dev\Cxx\src\Cxx.jl:178
crt_sig_handler at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\signals-win.c:91
raise at C:\WINDOWS\System32\msvcrt.dll (unknown line)
abort at C:\WINDOWS\System32\msvcrt.dll (unknown line)
_wassert at C:\Users\Maxime\.julia\dev\Cxx\deps\usr\bin\libcxxffi.DLL (unknown line)

Gnimuc commented 5 years ago

Hi @MaximeBouton, did you download the corresponding Julia and LLVM/Clang binaries and unzip to the correct folder?

MaximeBouton commented 5 years ago

I just edited my message before seeing your answer, that was the issue I think. I downloaded the windows version of julia and this version of llvm: https://github.com/staticfloat/LLVMBuilder/releases/download/v6.0.1-5%2Bnowasm/LLVM.v6.0.1.x86_64-w64-mingw32-gcc7.tar.gz

Building works but I am getting an issue when calling using:

julia> using Cxx

[ Info: Precompiling Cxx [a0b5b9ef-44b7-5148-a2d1-f6db19f3c3d2]
A s s e r t i o n   f a i l e d !

 P r o g r a m :   C : \ P r o g r a m   F i l e s \ J u l i a - 1 . 1 . 0 \ b i n \ j u l i a . e x e
 F i l e :   / w o r k s p a c e / s r c d i r / b o o t s t r a p . c p p ,   L i n e   1 5 2 3

 E x p r e s s i o n :   f _ j u l i a _ t y p e _ t o _ l l v m

signal (22): SIGABRT
in expression starting at C:\Users\Maxime\.julia\dev\Cxx\src\Cxx.jl:178
crt_sig_handler at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\signals-win.c:91
raise at C:\WINDOWS\System32\msvcrt.dll (unknown line)
abort at C:\WINDOWS\System32\msvcrt.dll (unknown line)
_wassert at C:\Users\Maxime\.julia\dev\Cxx\deps\usr\bin\libcxxffi.DLL (unknown line)

Gnimuc commented 5 years ago

Hi @MaximeBouton, sorry for the delayed response. The LLVMBuilder version that I used was actually v6.0.1-4, not v6.0.1-5. However, I thought the error was not likely caused by the mismatched version. Instead, that assertion error meant that Cxx was not able to access function pointers from Julia runtime, which was very likely due to my buggy patch here. Then, I tried a more robust opensource solution dlfcn-win32(v1.1.2), but no luck. The following example still crashed Julia, which, I suspected, was also due to that assertion.

julia> using Cxx
julia> cxx""" #include<iostream> """
true

julia> julia_global = 1
1

julia> cxx"""
       uint64_t bar() {
           return (uint64_t)$:(julia_global::Int64);
           }
           """

As dlfcn-win32 tagged a new release 3 days ago, I spot there is a bugfix about using dlsym() with RTLD_DEFAULT, so I just tried it and now it turns out the above example works normally. I updated the build scripts in a new branch here and added a step-by-step build doc for Windows at the bottom of README. Other Windows issues are probably also due to my buggy commits, for example, this one which I guess is the root cause of

julia> @cxx printme(pointer("John"))
LLVM ERROR: Program used external function '__gxx_personality_v0' which could not be resolved!

Gnimuc commented 5 years ago

~~Some tests that don't get passed on Windows are listed below:~~

UPDATE:

Almost all tests passed except these exception handling tests.

# Exception handling
try
    icxx" throw 20; "
    @assert false
catch e
    buf = IOBuffer();
    showerror(buf,e)
    @assert String(take!(buf)) == "20"
end

cxx"""
class test_exception : public std::exception
{
public:
    int x;
    test_exception(int x) : x(x) {};
};
"""

import Base: showerror
@exception function showerror(io::IO, e::rcpp"test_exception")
    print(io, icxx"$e.x;")
end

try
    icxx" throw test_exception(5); "
    @assert false
catch e
    buf = IOBuffer();
    showerror(buf,e)
    @assert String(take!(buf)) == "5"
end

azurefx commented 5 years ago

Tested on v0.3.2 and master=93359f1

(v1.1) pkg> add https://github.com/JuliaInterop/Cxx.jl#master
  Updating git-repo `https://github.com/JuliaInterop/Cxx.jl`
 Resolving package versions...
  Updating `C:\Users\Azure\.julia\environments\v1.1\Project.toml`
  [a0b5b9ef]   Cxx v0.3.2 #master (https://github.com/JuliaInterop/Cxx.jl)
  Updating `C:\Users\Azure\.julia\environments\v1.1\Manifest.toml`
  [a0b5b9ef]   Cxx v0.3.2 #master (https://github.com/JuliaInterop/Cxx.jl)
  Building Cxx → `C:\Users\Azure\.julia\packages\Cxx\vxYtJ\deps\build.log`

julia> using Cxx
[ Info: Recompiling stale cache file C:\Users\Azure\.julia\compiled\v1.1\Cxx\ESGkI.ji for Cxx [a0b5b9ef-44b7-5148-a2d1-f6db19f3c3d2]

C++ >(any input) LLVM ERROR: Program used external function '_ZN5clang5LexerC1ENS_14SourceLocationERKNS_11LangOptionsEPKcS6_S6_' which could not be resolved!

julia> versioninfo()
Julia Version 1.1.0
Commit 80516ca202 (2019-01-21 21:24 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-4712MQ CPU @ 2.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, haswell)
Environment:
  JULIA_NUM_THREADS = 4

azurefx commented 5 years ago

julia> using Cxx

julia> cxx"#include <cstdio>"
true

julia> cxx"""void f(double x){
       printf("%f\n",x);
       }"""
true

julia> @cxx f(typemax(Float64))
LLVM ERROR: Program used external function '__mingw_vprintf' which could not be resolved!

Gnimuc commented 5 years ago

hi @phlavenk, __gxx_personality_v0 related issues has been ~~partially~~ fixed ~~by a ugly workaround here. Also, you need to preserve the temporary pointer when calling @cxx printme(ptr)~~:

julia> ptr = pointer("John")
Ptr{UInt8} @0x000000000a8b8cd8

julia> GC.@preserve ptr @cxx printme(ptr)

Gnimuc commented 5 years ago

LLVM ERROR: Program used external function '__mingw_vprintf' which could not be resolved!

@azurefx this looks like those libstdc++ symbols are still not correctly exported and ~~the old __gxx_personality_v0 issues could also be caused by this.~~ At least those "LLVM/Clang symbol missing" errors are now fixed in PR #425 which makes Windows REPL working again.

chandran-saravanan commented 5 years ago

I added Cxx package, I successfully executed using Cxx, but when I try with #include , the Windows PowerShell quickly closes automatically showing few text lines which are unable to read.

Gnimuc commented 5 years ago

@chandran-saravanan the latest release of Cxx doesn't support Cxx REPL on Windows, please checkout PR #425 for the fix.

JuliaInterop / Cxx.jl

Windows issues #409

UPDATE: