julia_init() crashes R

[mkln]

Hi. I have successfully installed rjulia on 3 computers (2x Windows10, 1x Windows7), and I can run library(rjulia) with no error. Note that I had originally tried installing through Rstudio (latest version) but compilation failed. running from R base seemed to solve that issue.

I have julia 0.4.1 and R 3.2.2.

in all computers, running julia_init() makes R crash (same if I run R from Rstudio).

Since this issue appears in 3 different computers, Im wondering what's going on

[felipenoris]

same here... error message:

Exception: EXCEPTION_ACCESS_VIOLATION at 0x0 -- unknown function (ip: 0000000000000000)

[felipenoris]

I guess it´s a general problem: https://github.com/JuliaLang/julia/issues/13570

[armgong]

maybe this issue is caused by julia, give some try: on linux and julia 0.4.1 run ok on windows and julia 0.4.1 crash on windows and julia 0.4.1 pre ok

library(rjulia) julia_init() julia_void_eval("print(versioninfo())") Julia Version 0.4.1-pre+22 Commit 669222e* (2015-11-01 00:06 UTC) Platform Info: System: Windows (x86_64-w64-mingw32) CPU: Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz WORD_SIZE: 64 BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NOAFFINITY Haswell) LAPACK: libopenblas64 LIBM: libopenlibm LLVM: libLLVM-3.3

so this is issue caused by julia 0.4.1 , I will find some time to trace which commit caused this issue, at least Commit 669222e* is ok , and 0.4.1 is wrong .

as a temporary fix please use julia 0.4.0 or some version before Commit 669222e*

[mkln]

I have installed julia 0.4.0, changed the path environmental variable, and reinstalled rjulia through devtools::install_github("armgong/rjulia", ref="master", args = "--no-multiarch")

but the same problem appears (which by the way is the same as reported by felipenoris)

but maybe I'm missing some steps?

[armgong]

I just compile julia 0.4.1 and rjulia on Win10 64bit, julia_init is ok. so it is strange. when use julia 0.4.1 download from julialang.org ,and compile rjulia use msys2+mingwbuild 4.92, it will crash R. but when use julia 0.4.1 compile by myself , everything is ok.

could you try compile julia and rjulia use msys2+mingwbuild 4.92 rev3 ? and tell me what happends?

gcc version 4.9.2 (x86_64-win32-seh-rev3, Built by MinGW-W64 project) Thread model: win32

[armgong]

Or could you try just compile rjulia use tool-chain that julialang used (https://github.com/JuliaLang/julia/blob/master/contrib/windows/get_toolchain.sh) ? and tell me what happends?

don't use Rtools toolchain ,just use julialang toolchain. and only need compile rjulia

[armgong]

after test some combine, found the only way to work on windows is: 1 compile and install julia use msys2+mingw-build ,don't use rtools or julialang toolchain. 2 rjulia can use msys2+mingw-build or rtools or julialang toolchain to compile to install.

[mkln]

I'm trying to build julia, but I get an error. I have no idea what the root of this error is.

$ make -j 1
patching file CMakeLists.txt
Hunk #1 FAILED at 217.
1 out of 1 hunk FAILED -- saving rejects to file CMakeLists.txt.rej
-- The C compiler identification is GNU 4.9.2
-- Check for working C compiler: C:/mingw64/bin/gcc.exe
-- Check for working C compiler: C:/mingw64/bin/gcc.exe -- broken
CMake Error at C:/CMake/share/cmake-3.4/Modules/CMakeTestCCompiler.cmake:61 (message):
  The C compiler "C:/mingw64/bin/gcc.exe" is not able to compile a simple
  test program.

  It fails with the following output:

   Change Dir: C:/msys64/home/peruz/julia/deps/build/libgit2/CMakeFiles/CMakeTmp

  Run Build Command:"C:/msys64/usr/bin/make.exe" "cmTC_eeaeb/fast"

  make[2]: Entering directory
  '/home/peruz/julia/deps/build/libgit2/CMakeFiles/CMakeTmp'

  /usr/bin/make -f CMakeFiles/cmTC_eeaeb.dir/build.make
  CMakeFiles/cmTC_eeaeb.dir/build

        0 [main] make 9836 C:\msys64\usr\bin\make.exe: *** fatal error in forked process - fork: can't reserve memory for parent stack 0x600000 - 0x800000, (child has 0x400000 - 0x600000), Win32 error 487
      404 [main] make 9836 cygwin_exception::open_stackdumpfile: Dumping stack trace to make.exe.stackdump
        1 [main] make 7300 fork: child -1 - forked process 9836 died unexpectedly, retry 0, exit code 0x100, errno 11

  make[2]: fork: Resource temporarily unavailable

  make[2]: Leaving directory
  '/home/peruz/julia/deps/build/libgit2/CMakeFiles/CMakeTmp'

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:14 (PROJECT)

-- Configuring incomplete, errors occurred!
See also "C:/msys64/home/peruz/julia/deps/build/libgit2/CMakeFiles/CMakeOutput.log".
See also "C:/msys64/home/peruz/julia/deps/build/libgit2/CMakeFiles/CMakeError.log".
Makefile:2035: recipe for target 'build/libgit2/Makefile' failed
make[1]: *** [build/libgit2/Makefile] Error 1
Makefile:78: recipe for target 'julia-deps' failed
make: *** [julia-deps] Error 2

[armgong]

I build julia-0.4.1 for win64, you can download https://github.com/armgong/julia-64-build-with-mingw-builds-for-R ,and try it.

[mkln]

Ok, thanks a lot! julia_init() doesnt crash R this way anymore.

However I tried to demo testdataframe.R and it says

Warning message:
In jloaddf() : DataArray and DataFrame Julia packages have not been loaded.
  Please install or check installation directory.

but from Julia doing using DataFrames works fine. Ideas?

[armgong]

1 julia_void_eval("print(LOAD_PATH)") in R console, and see the output

Union{ASCIIString,UTF8String} ["C:\julia\64\local\share\julia\site\v0.4","C:\julia\64\share\julia\site\v0.4","C:\Users\armgong.julia\v0.4"] ，the last item is important.

2 try julia_void_eval("using DataFrames") in R console.

this will recompile the package to C:\Users\armgong\documents.julia\v0.4 , this is because R set HOMEDIR as users\username\documents but julia set HOMEDIR as users\username, so when first using a package in rjulia, julia will recompile .ji to users\username\documents, because R set HOMEDIR different with julia, and this inevitable.

btw,on my machine all code is just fine

[armgong]

also make a patch force ENV["HOME"] to julia's ,may affect other package, but let we try this first. you can install new version rjulia and see what happens.

[mkln]

So here's what happens:

On R (not Rstudio) all the commands give no output, like so: (but running the "using DataFrames" line makes a julia empty prompt appear, then it goes away.

> library(rjulia)
> julia_init()
> julia_eval("1+1")
[1] 2
> julia_void_eval("print(LOAD_PATH)")
> julia_void_eval("println(LOAD_PATH)")
> julia_void_eval("using DataFrames") 

> 

On Rstudio instead

> library(rjulia)
> julia_init()
> julia_eval("1+1")
[1] 2
> julia_void_eval("print(LOAD_PATH)")
Union{ASCIIString,UTF8String}["C:\\Users\\peruz\\julia\\local\\share\\julia\\site\\v0.4","C:\\Users\\peruz\\julia\\share\\julia\\site\\v0.4","C:\\Users\\peruz\\.julia\\v0.4"]
> julia_void_eval("using DataFrames")
ERROR: LoadError: LoadError: error compiling anonymous: could not load library "zlib1"
The specified module could not be found.

 in include at boot.jl:261
 in include_from_node1 at loading.jl:304
 in include at boot.jl:261
 in include_from_node1 at loading.jl:304
 [inlined code] from none:2
 in anonymous at no file:0
 in process_options at client.jl:257
 in _start at client.jl:378
while loading C:\Users\peruz\.julia\v0.4\GZip\src\zlib_h.jl, in expression starting on line 8
while loading C:\Users\peruz\.julia\v0.4\GZip\src\GZip.jl, in expression starting on line 75
ERROR: LoadError: Failed to precompile GZip to C:\Users\peruz\Documents\.julia\lib\v0.4\GZip.ji
 in error at error.jl:21
while loading C:\Users\peruz\.julia\v0.4\DataFrames\src\DataFrames.jl, in expression starting on line 15
ErrorException("Failed to precompile DataFrames to C:\\Users\\peruz\\Documents\\.julia\\lib\\v0.4\\DataFrames.ji")

[armgong]

this mean on bare R console ,everything is ok, you can run demo code. but Rstudio whichi can't find zlib1.dll. I forgot it ,now upload it to https://github.com/armgong/julia-64-build-with-mingw-builds-for-R, and copy it to your PATH (like julia\bin or R\bin\x64 or R\bin)

[mkln]

So, the zlib1.dll you uploaded didn't work -- "not a valid win32 application" I don't know the reason, but since it was supposed to be in julia\bin then I used the one I had from a juno folder. That worked.

Now I found another issue -- I don't know if it's an issue yet as I'm just starting to write R code with julia chunks. I dont understand this behavior:

> j2r("k")
[1] 3
> j2r("zeros(3)")
[1] 0 0 0
> j2r("zeros(k)")
[1] 0

the same happens with the ones function. I would have expected the same results.

Then one other thing: I have written a simple gibbs sampler in julia the output of which is just a Dataframe:

julia> typeof(mc_data)
DataFrames.DataFrame
julia> size(mc_data)
(10000,6)

However, the code below makes R crash:

jDo('include("C:/path/to/file/gibbs_julia_slave.jl")')
mcdata <- j2r("mc_data")

More specifically, it crashes after running the whole loop, at the request for "mc_data".

Thanks for the help!

[armgong]

could you export mc_data to csv or txt , then I can download and use it to debug

[mkln]

I traced the problem back to convert in julia

This makes R crash:

j2r("convert(DataFrame, [1 2 3; 4 5 6])")

any ideas for the zeros(k) issue?

[armgong]

1 zeros(k) issue, on my system is ok. maybe you misspelling zeros to zero? zeros[k] is 0 0 0 but zero[3] will 0 ,could your recheck the code? I test on windows and linux, both system run code ok.

[armgong]

and data frame issue now is fixed, please reinstall rjulia.

[mkln]

Here:

> r2j(3, "k")
> j2r("k")
[1] 3
> j2r("ones(k)")
[1] 1
> julia_void_eval("k = 3")
> j2r("ones(k)")
[1] 1 1 1

Thanks for the fix with the dataframes!

[armgong]

finally I found the reason about the issue, it is not a bug, just convertion rules. rjulia with translate k=3 in R to julia 1-dim array , k in julia is an array which length is 1. r2j(3, "k") j2r("ones(k[1])") will be ok.

[armgong]

to verify this, you can run following code in julia:

k=[3] ones[k] ones[k[1]]

[mkln]

Yep, I see what you mean. Great so this is also solved.

Btw, will you provide a complied julia on a regular basis?

[mkln]

Ok so actually I have a julia_init() problem even with the "new" julia. But this only happens in a surface pro 4 (hence intel core 6th gen), whereas it does not happen in 3 different computers with 4th gen processors (among which a surface pro 3) the cpu is the first thing I thought of, I used the same procedure with all.

Suggestions?

[armgong]

did you try rename libopenblas_64.dll.0 to libopenblas_64.dll in my build? maybe caused by it??

[mkln]

I've now tried to rename libopenblas64.dll to libopenblas64.dll.old and viceversa the one named libopenblas64.dll.old to libopenblas64.dll but the effect is the same. The error shown in Rstudio is the same as felipenoris reported.