JuliaLang / julia

The Julia Programming Language
https://julialang.org/
MIT License
45.92k stars 5.49k forks source link

Julia crashes when running Pkg.update() #34785

Closed unsoundsystem closed 4 years ago

unsoundsystem commented 4 years ago

Segfault occur when updating julia packages. The problem happen after update of archlinux.

julia> versioninfo()
Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

error message

signal (11): Segmentation fault
in expression starting at REPL[2]:1
unknown function (ip: 0x7f560d249e73)
http_parser_execute at /usr/bin/../lib/libhttp_parser.so.2.9 (unknown line)
unknown function (ip: 0x7f560d24b390)
unknown function (ip: 0x7f560d24d770)
unknown function (ip: 0x7f560d24fb58)
unknown function (ip: 0x7f560d24df91)
git_remote_fetch at /usr/bin/../lib/libgit2.so (unknown line)
git_clone at /usr/bin/../lib/libgit2.so (unknown line)
unknown function (ip: 0x7f563000597f)
unknown function (ip: 0x7f5630006991)
unknown function (ip: 0x7f563003ae5e)
unknown function (ip: 0x7f563003b3b9)
unknown function (ip: 0x7f563003b53e)
unknown function (ip: 0x7f563003bdda)
unknown function (ip: 0x7f563003db39)
#up#43 at /build/julia/src/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:152
up at /build/julia/src/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:146 [inlined]
#up#42 at /build/julia/src/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:141 [inlined]
up at /build/julia/src/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:141 [inlined]
#up#39 at /build/julia/src/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:138 [inlined]
up at /build/julia/src/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:138
unknown function (ip: 0x7f56162ec12c)
unknown function (ip: 0x7f563c4feced)
unknown function (ip: 0x7f563c4fe99a)
unknown function (ip: 0x7f563c4ff6d4)
unknown function (ip: 0x7f563c4ffdf9)
unknown function (ip: 0xfffffffffffffffe)
unknown function (ip: 0x7f562e86d60f)
unknown function (ip: (nil))
unknown function (ip: 0x7f563c500219)
unknown function (ip: 0x7f563c3c44fd)
unknown function (ip: 0x7f563c3c4b88)
unknown function (ip: 0x7f563c3c4b88)
jl_toplevel_eval_in at /usr/bin/../lib/libjulia.so.1 (unknown line)
unknown function (ip: 0x7f562ff06c64)
unknown function (ip: 0x7f56300f4659)
unknown function (ip: 0x7f56300f4954)
unknown function (ip: 0x7f563c3aac94)
unknown function (ip: 0xffffffffffffffff)
Allocations: 954860 (Pool: 954578; Big: 282); GC: 1
Segmentation fault (core dumped)
KristofferC commented 4 years ago

This looks the same as https://discourse.julialang.org/t/pkg-crashes/34750/3.

Solution is to use the official binaries at https://julialang.org/downloads/. Please comment if this doesn't fix the problem.

unsoundsystem commented 4 years ago

Fixed thank you very much!!!

alyst commented 4 years ago

Using an official binary could be a solution, but I'm using Julia built with MKL, so I need a way to compile it with the recent versions of the OS packages. There's a corresponding ArchLinux Issue #65540. Eventually, it would be nice to fix it in the official package. For me it looks like http-parser-2.9.3 requires a newer version of libgit2. Which, in turn, requires an update to stdlib/LibGit2. Maybe some of the Julia devs can provide more details on the issue, so that the efforts in #65540 would be more efficient?

KristofferC commented 4 years ago

MKL.jl should work with the official binaries.

alyst commented 4 years ago

Thanks for the suggestion, I was not aware of MKL.jl! Now I'm trying to use julia-bin AUR package, which provides the official 1.3.1 binaries. However, using MKL.jl "as is" is a no-go for me, as it needs to update /usr/.../julia/base/sysimg.jl managed by the OS package manager. I'm now trying to use https://github.com/tkf/JuliaManager.jl to have a system image in my home folder, but this package is not up-to-date and conflicts with MKL.jl (because of PackageCompiler.jl). I can try fixing it myself (but I see that it's already more than just fixing Project.toml), unless there's better solution.

KristofferC commented 4 years ago

I meant just downloading and using the official binaries without a package manager. It might be possible for PackageCompiler to not require write access to that folder though.

alyst commented 4 years ago

I think I understand how supporting rolling release Linux distributions looks from the Julia developer perspective, but I would really like to avoid manually maintaining multiple software packages for multiple users in their home folders.

I'm trying now to automate the process of building MKL-enabled system image. Here's an excerpt from the modified julia-bin AUR (${srcdir}/julia-${pkgver} is where the official binaries are extracted):

build() {
  # rebuild sysimage with MKL support
  rm -rf julia-${pkgver}/tmpdepot
  mkdir -p julia-${pkgver}/tmpdepot/tmpenv
  cd julia-${pkgver}/tmpdepot/tmpenv
  JULIA_DEPOT_PATH=${srcdir}/julia-${pkgver}/tmpdepot ${srcdir}/julia-${pkgver}/bin/julia \
    -H ${srcdir}/julia-${pkgver}/bin \
    --startup-file=no \
    --project=. \
    ${srcdir}/build_sysimg_with_mkl.jl
}

and here's build_sysimg_with_mkl.jl that gets called:

using Pkg
Pkg.add(PackageSpec(url="https://github.com/JuliaComputing/MKL.jl"))

However, I'm getting this error:

...
  Building MKL → `~/aur/julia-mkl-bin/src/julia-1.3.1/tmpdepot/packages/MKL/1edTo/deps/build.log`
┌ Error: Error building `MKL`: 
│ error during bootstrap:
│ LoadError("/tmp/jl_8dlP7m/sysimage_packagecompiler_4630637a-614a-11ea-16df-07de228ef25d.jl", 16, ArgumentError("Package MKL not found in current path:\n- Run `import Pkg; Pkg.add(\"MKL\")` to install the MKL package.\n"))
│ rec_backtrace at /buildworker/worker/package_linux64/build/src/stackwalk.c:94
│ record_backtrace at /buildworker/worker/package_linux64/build/src/task.c:224 [inlined]
│ jl_throw at /buildworker/worker/package_linux64/build/src/task.c:461
│ require at ./loading.jl:887
.... <skip until the end of the log>
│ [ Info: Splicing in code to load MKL in /home/theuser/aur/julia-mkl-bin/src/julia-1.3.1/bin/../share/julia/base/sysimg.jl
│ [ Info: getting precompile script from: https://raw.githubusercontent.com/JuliaLang/julia/release-1.3/contrib/generate_precompile.jl
│ [ Info: PackageCompiler: creating base system image (incremental=false)...
│ ERROR: LoadError: failed process: Process(`/home/theuser/aur/julia-mkl-bin/src/julia-1.3.1/bin/julia --color=yes --startup-file=no --cpu-target native --sysimage=/tmp/jl_8dlP7m/corecompiler.ji -g1 -O0 --output-ji=/tmp/jl_8dlP7m/sys.ji /tmp/jl_8dlP7m/sysimage_packagecompiler_4630637a-614a-11ea-16df-07de228ef25d.jl`, ProcessExited(1)) [1]
│ 
│ Stacktrace:
│  [1] pipeline_error at ./process.jl:525 [inlined]
│  [2] read(::Cmd) at ./process.jl:412
│  [3] (::PackageCompiler.var"#2#3"{String,Array{String,1},String,String,String,String})() at /home/theuser/aur/julia-mkl-bin/src/julia-1.3.1/tmpdepot/packages/PackageCompiler/D8oaG/src/PackageCompiler.jl:157
│  [4] cd(::PackageCompiler.var"#2#3"{String,Array{String,1},String,String,String,String}, ::String) at ./file.jl:104
│  [5] #create_fresh_base_sysimage#1(::String, ::typeof(PackageCompiler.create_fresh_base_sysimage), ::Array{String,1}) at /home/theuser/aur/julia-mkl-bin/src/julia-1.3.1/tmpdepot/packages/PackageCompiler/D8oaG/src/PackageCompiler.jl:141
│  [6] #create_fresh_base_sysimage at ./tuple.jl:0 [inlined]
│  [7] #create_sysimage#5(::Nothing, ::String, ::Array{String,1}, ::Array{String,1}, ::Bool, ::Bool, ::Bool, ::String, ::String, ::Nothing, ::Bool, ::typeof(create_sysimage), ::Array{Symbol,1}) at /home/theuser/aur/julia-mkl-bin/src/julia-1.3.1/tmpdepot/packages/PackageCompiler/D8oaG/src/PackageCompiler.jl:414
│  [8] (::PackageCompiler.var"#kw##create_sysimage")(::NamedTuple{(:incremental, :replace_default, :script),Tuple{Bool,Bool,String}}, ::typeof(create_sysimage), ::Array{Symbol,1}) at ./none:0 (repeats 2 times)
│  [9] top-level scope at /home/theuser/aur/julia-mkl-bin/src/julia-1.3.1/tmpdepot/packages/MKL/1edTo/src/install.jl:107
│  [10] eval at ./boot.jl:330 [inlined]
│  [11] change_blas_library(::String) at /home/theuser/aur/julia-mkl-bin/src/julia-1.3.1/tmpdepot/packages/MKL/1edTo/src/install.jl:105
│  [12] enable_mkl_startup() at /home/theuser/aur/julia-mkl-bin/src/julia-1.3.1/tmpdepot/packages/MKL/1edTo/src/install.jl:82
│  [13] top-level scope at /home/theuser/aur/julia-mkl-bin/src/julia-1.3.1/tmpdepot/packages/MKL/1edTo/deps/build.jl:5
│  [14] include at ./boot.jl:328 [inlined]
│  [15] include_relative(::Module, ::String) at ./loading.jl:1105
│  [16] include(::Module, ::String) at ./Base.jl:31
│  [17] include(::String) at ./client.jl:424
│  [18] top-level scope at none:5
│ in expression starting at /home/theuser/aur/julia-mkl-bin/src/julia-1.3.1/tmpdepot/packages/MKL/1edTo/deps/build.jl:5
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/backwards_compatible_isolation.jl:649

Do I miss something very simple?