search

JuliaNLSolvers / NLsolve.jl

Julia solvers for systems of nonlinear equations and mixed complementarity problems
Other
326 stars 66 forks source link

Segmentation fault #123

Closed blegat closed 6 years ago

blegat commented 6 years ago

I get a segmentation fault when running the first example:

using NLsolve

function f!(x, fvec)
    fvec[1] = (x[1]+3)*(x[2]^3-7)+18
    fvec[2] = sin(x[2]*exp(x[1])-1)
end

function g!(x, fjac)
    fjac[1, 1] = x[2]^3-7
    fjac[1, 2] = 3*x[2]^2*(x[1]+3)
    u = exp(x[1])*cos(x[2]*exp(x[1])-1)
    fjac[2, 1] = x[2]*u
    fjac[2, 2] = u
end

nlsolve(f!, g!, [0.1; 1.2])

I get the following

signal (11): Segmentation fault
while loading /home/blegat/tmp.jl, in expression starting on line 16
unknown function (ip: 0x7f56a756a761)
unknown function (ip: 0x7f56a756c5d4)
_ZN4llvm12RegAllocBase16postOptimizationEv at /usr/bin/../lib/libLLVM-3.9.so (unknown line)
unknown function (ip: 0x7f56a76bb1d7)
_ZN4llvm19MachineFunctionPass13runOnFunctionERNS_8FunctionE at /usr/bin/../lib/libLLVM-3.9.so (unknown line)
_ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE at /usr/bin/../lib/libLLVM-3.9.so (unknown line)
_ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE at /usr/bin/../lib/libLLVM-3.9.so (unknown line)
_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /usr/bin/../lib/libLLVM-3.9.so (unknown line)
unknown function (ip: 0x7f56aa7e399d)
unknown function (ip: 0x7f56aa7e934f)
unknown function (ip: 0x7f56aa7e9e27)
unknown function (ip: 0x7f56aa79c9b1)
unknown function (ip: 0x7f56aa7d9c07)
jl_apply_generic at /usr/bin/../lib/libjulia.so.0.6 (unknown line)
#nlsolve at ./<missing>:0
#nlsolve#13 at /home/blegat/.julia/v0.6/NLsolve/src/nlsolve_func_defs.jl:52
unknown function (ip: 0x7f56964e7772)
jl_apply_generic at /usr/bin/../lib/libjulia.so.0.6 (unknown line)
nlsolve at /home/blegat/.julia/v0.6/NLsolve/src/nlsolve_func_defs.jl:52
unknown function (ip: 0x7f56964e72cd)
jl_apply_generic at /usr/bin/../lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x7f56aa74c51f)
unknown function (ip: 0x7f56aa74b64e)
unknown function (ip: 0x7f56aa74c453)
unknown function (ip: 0x7f56aa764585)
unknown function (ip: 0x7f56aa740ec6)
jl_load at /usr/bin/../lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x7f56a2fedca5)
unknown function (ip: 0x7f56a2fede7b)
jl_apply_generic at /usr/bin/../lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x7f56a2e92d6f)
unknown function (ip: 0x7f56a2e92ddb)
jl_apply_generic at /usr/bin/../lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x7f56a2ff6ad5)
unknown function (ip: 0x7f56a2ff8fb4)
unknown function (ip: 0x7f56a2ff9a48)
jl_apply_generic at /usr/bin/../lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x5646b38c1c4b)
unknown function (ip: 0x5646b38c1652)
__libc_start_main at /usr/bin/../lib/libc.so.6 (unknown line)
unknown function (ip: 0x5646b38c1709)
Allocations: 34563700 (Pool: 34560695; Big: 3005); GC: 61

When running the MCP example:

using NLsolve

function f!(x, fvec)
    fvec[1]=3*x[1]^2+2*x[1]*x[2]+2*x[2]^2+x[3]+3*x[4]-6
    fvec[2]=2*x[1]^2+x[1]+x[2]^2+3*x[3]+2*x[4]-2
    fvec[3]=3*x[1]^2+x[1]*x[2]+2*x[2]^2+2*x[3]+3*x[4]-1
    fvec[4]=x[1]^2+3*x[2]^2+2*x[3]+3*x[4]-3
end

r = mcpsolve(f!, [0., 0., 0., 0.], [Inf, Inf, Inf, Inf],
             [1.25, 0., 0., 0.5], reformulation = :smooth, autodiff = true)

I get another segmentation fault

signal (11): Segmentation fault
while loading /home/blegat/mcp.jl, in expression starting on line 10
unknown function (ip: 0x7f8d657e8761)
unknown function (ip: 0x7f8d657ea5d4)
_ZN4llvm12RegAllocBase16postOptimizationEv at /usr/bin/../lib/libLLVM-3.9.so (unknown line)
unknown function (ip: 0x7f8d659391d7)
_ZN4llvm19MachineFunctionPass13runOnFunctionERNS_8FunctionE at /usr/bin/../lib/libLLVM-3.9.so (unknown line)
_ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE at /usr/bin/../lib/libLLVM-3.9.so (unknown line)
_ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE at /usr/bin/../lib/libLLVM-3.9.so (unknown line)
_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /usr/bin/../lib/libLLVM-3.9.so (unknown line)
unknown function (ip: 0x7f8d68a6199d)
unknown function (ip: 0x7f8d68a6734f)
unknown function (ip: 0x7f8d68a67e27)
unknown function (ip: 0x7f8d68a1a9b1)
unknown function (ip: 0x7f8d68a57c07)
jl_apply_generic at /usr/bin/../lib/libjulia.so.0.6 (unknown line)
#nlsolve at ./<missing>:0
unknown function (ip: 0x7f8d547696dd)
jl_apply_generic at /usr/bin/../lib/libjulia.so.0.6 (unknown line)
#mcpsolve#17 at /home/blegat/.julia/v0.6/NLsolve/src/mcp_func_defs.jl:84
unknown function (ip: 0x7f8d547668e4)
jl_apply_generic at /usr/bin/../lib/libjulia.so.0.6 (unknown line)
#mcpsolve at ./<missing>:0
unknown function (ip: 0x7f8d54765fb6)
jl_apply_generic at /usr/bin/../lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x7f8d689ca51f)
unknown function (ip: 0x7f8d689c964e)
unknown function (ip: 0x7f8d689cad28)
unknown function (ip: 0x7f8d689cb4a8)
unknown function (ip: 0x7f8d689e2b1d)
unknown function (ip: 0x7f8d689beec6)
jl_load at /usr/bin/../lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x7f8d6126bca5)
unknown function (ip: 0x7f8d6126be7b)
jl_apply_generic at /usr/bin/../lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x7f8d61110d6f)
unknown function (ip: 0x7f8d61110ddb)
jl_apply_generic at /usr/bin/../lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x7f8d61274ad5)
unknown function (ip: 0x7f8d61276fb4)
unknown function (ip: 0x7f8d61277a48)
jl_apply_generic at /usr/bin/../lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x55b9fe506c4b)
unknown function (ip: 0x55b9fe506652)
__libc_start_main at /usr/bin/../lib/libc.so.6 (unknown line)
unknown function (ip: 0x55b9fe506709)
Allocations: 35242078 (Pool: 35238992; Big: 3086); GC: 62

I am using version v0.12.1 of NLsolve and version v0.6.0 of Julia.

pkofod commented 6 years ago

Sorry, I didn't see it (apparently I wasn't "watching" this repo...) I'll look into it!

Edit: did this also happen earlier? I wonder if we can find a simpler example/find out what operation causes this.

stakaz commented 6 years ago

Hello, this issue is still present for me in julia 0.6.2. Trying the first example leads to:

signal (11): Segmentation fault
while loading /media/DATA/PhD/code/sandbox/test_nlsolve.jl, in expression starting on line 16
unknown function (ip: 0x7f1aab306761)
unknown function (ip: 0x7f1aab3085d4)
_ZN4llvm12RegAllocBase16postOptimizationEv at /usr/lib/libLLVM-3.9.so (unknown line)
unknown function (ip: 0x7f1aab4571d7)
_ZN4llvm19MachineFunctionPass13runOnFunctionERNS_8FunctionE at /usr/lib/libLLVM-3.9.so (unknown line)
_ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE at /usr/lib/libLLVM-3.9.so (unknown line)
_ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE at /usr/lib/libLLVM-3.9.so (unknown line)
_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /usr/lib/libLLVM-3.9.so (unknown line)
unknown function (ip: 0x7f1aae57f5de)
unknown function (ip: 0x7f1aae585033)
unknown function (ip: 0x7f1aae585b29)
unknown function (ip: 0x7f1aae537362)
unknown function (ip: 0x7f1aae575527)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
#nlsolve at ./<missing>:0
#nlsolve#13 at /home/gluon/.julia/v0.6/NLsolve/src/nlsolve_func_defs.jl:52
unknown function (ip: 0x7f1a98063302)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
nlsolve at /home/gluon/.julia/v0.6/NLsolve/src/nlsolve_func_defs.jl:52
unknown function (ip: 0x7f1a98062e5d)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x7f1aae4e717f)
unknown function (ip: 0x7f1aae4e62ae)
unknown function (ip: 0x7f1aae4e70b3)
unknown function (ip: 0x7f1aae4ff245)
unknown function (ip: 0x7f1aae4dbb16)
unknown function (ip: 0x7f1aa6949b1b)
unknown function (ip: 0x7f1aa6949c0f)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x7f1aae4e717f)
unknown function (ip: 0x7f1aae4e62ae)
unknown function (ip: 0x7f1aae4e70b3)
unknown function (ip: 0x7f1aae4fe096)
jl_toplevel_eval_in at /usr/lib/libjulia.so.0.6 (unknown line)
include_string at /home/gluon/.julia/v0.6/Compat/src/Compat.jl:174
unknown function (ip: 0x7f1a98058add)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
#57 at /home/gluon/.julia/v0.6/Atom/src/eval.jl:74
withpath at /home/gluon/.julia/v0.6/CodeTools/src/utils.jl:30
unknown function (ip: 0x7f1a980580c6)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
withpath at /home/gluon/.julia/v0.6/Atom/src/eval.jl:38
unknown function (ip: 0x7f1a98057d86)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
macro expansion at /home/gluon/.julia/v0.6/Atom/src/eval.jl:72 [inlined]
#56 at ./task.jl:80
unknown function (ip: 0x7f1a98056d3f)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x7f1aae4ec4f9)
unknown function (ip: 0xffffffffffffffff)
Allocations: 34938135 (Pool: 34933300; Big: 4835); GC: 72

Does someone has an idea what to do to fix it?

pkofod commented 6 years ago

What's your NLsolve version and did you remember to take into account that we've flipped the argument order as described in the readme?

stakaz commented 6 years ago

I have used 0.13.0 for the test but I will now update to 0.14.1 and test it again with the right? order f!(F, x).

stakaz commented 6 years ago

Sitll the same with 0.14.1.

Here the test code:

using NLsolve

fvec = [0.0,0.0]

function f!(fvec,x)
    fvec[1] = (x[1]+3)*(x[2]^3-7)+18
    fvec[2] = sin(x[2]*exp(x[1])-1)
end

nlsolve(f!, [0.1; 1.2], autodiff = :forward)

Here the error

signal (11): Segmentation fault
while loading no file, in expression starting on line 0
unknown function (ip: 0x7f99dd892761)
unknown function (ip: 0x7f99dd8945d4)
_ZN4llvm12RegAllocBase16postOptimizationEv at /usr/lib/libLLVM-3.9.so (unknown line)
unknown function (ip: 0x7f99dd9e31d7)
_ZN4llvm19MachineFunctionPass13runOnFunctionERNS_8FunctionE at /usr/lib/libLLVM-3.9.so (unknown line)
_ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE at /usr/lib/libLLVM-3.9.so (unknown line)
_ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE at /usr/lib/libLLVM-3.9.so (unknown line)
_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /usr/lib/libLLVM-3.9.so (unknown line)
unknown function (ip: 0x7f99e0b0b5de)
unknown function (ip: 0x7f99e0b11033)
unknown function (ip: 0x7f99e0b11b29)
unknown function (ip: 0x7f99e0ac3362)
unknown function (ip: 0x7f99e0b01527)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
#nlsolve at ./<missing>:0
unknown function (ip: 0x7f99c4109f5d)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
#nlsolve#39 at /home/gluon/.julia/v0.6/NLsolve/src/nlsolve/nlsolve.jl:59
unknown function (ip: 0x7f99c4107415)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
#nlsolve at ./<missing>:0
unknown function (ip: 0x7f99c4106dfd)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x7f99e0a7317f)
unknown function (ip: 0x7f99e0a722ae)
unknown function (ip: 0x7f99e0a730b3)
unknown function (ip: 0x7f99e0a8a096)
jl_toplevel_eval_in at /usr/lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x7f99d8d11d99)
unknown function (ip: 0x7f99d8d11daf)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x7f99d8d92d81)
unknown function (ip: 0x7f99d8d92f9f)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
macro expansion at ./REPL.jl:97 [inlined]
#1 at ./event.jl:73
unknown function (ip: 0x7f99c40c234f)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x7f99e0a784f9)
unknown function (ip: 0xffffffffffffffff)
Allocations: 27754670 (Pool: 27750111; Big: 4559); GC: 53
Segmentation fault (core dumped)
pkofod commented 6 years ago

I straight up copied the code as you put it

julia> using NLsolve

julia> fvec = [0.0,0.0]
2-element Array{Float64,1}:
 0.0
 0.0

julia> function f!(fvec,x)
           fvec[1] = (x[1]+3)*(x[2]^3-7)+18
           fvec[2] = sin(x[2]*exp(x[1])-1)
       end
f! (generic function with 1 method)

julia> nlsolve(f!, [0.1; 1.2], autodiff = :forward)

Results of Nonlinear Solver Algorithm
 * Algorithm: Trust-region with dogleg and autoscaling
 * Starting Point: [0.1, 1.2]
 * Zero: [-3.7818e-16, 1.0]
 * Inf-norm of residuals: 0.000000
 * Iterations: 4
 * Convergence: true
   * |x - x'| < 0.0e+00: false
   * |f(x)| < 1.0e-08: true
 * Function Calls (f): 5
 * Jacobian Calls (df/dx): 5
pkofod commented 6 years ago

what's your versioninfo() ?

pkofod commented 6 years ago

And your Pkg.status()

stakaz commented 6 years ago 
julia> versioninfo()
Julia Version 0.6.2
Commit d386e40 (2017-12-13 18:08 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Pentium(R) CPU  N3530  @ 2.16GHz
  WORD_SIZE: 64
  BLAS: libopenblas (DYNAMIC_ARCH NO_AFFINITY Atom)
  LAPACK: libopenblas
  LIBM: libm
  LLVM: libLLVM-3.9.1 (ORCJIT, silvermont)
julia> Pkg.status()
28 required packages:
 - AmplNLWriter                  0.4.0
 - Atom                          0.6.6
 - BuildExecutable               0.1.2
 - CSV                           0.2.1
 - Clp                           0.3.1
 - Colors                        0.8.2
 - Combinatorics                 0.5.0
 - DataArrays                    0.7.0
 - DataFrames                    0.11.5
 - DecFP                         0.4.2
 - Formatting                    0.3.0              master
 - GSL                           0.3.6
 - Interpolations                0.7.3
 - Ipopt                         0.2.6
 - IterTools                     0.2.1
 - JLD                           0.8.3
 - JuMP                          0.18.0                                                                                                                                                                           
 - LaTeXStrings                  0.3.0                                                                                                                                                                            
 - Lint                          0.5.0+             master                                                                                                                                                        
 - LsqFit                        0.2.0                                                                                                                                                                            
 - NLopt                         0.3.6                                                                                                                                                                            
 - NLsolve                       0.14.1                                                                                                                                                                           
 - Optim                         0.12.0                                                                                                                                                                           
 - PGFPlots                      2.2.2                                                                                                                                                                            
 - Plots                         0.15.0                                                                                                                                                                           
 - PyPlot                        2.3.2                                                                                                                                                                            
 - RDatasets                     0.3.0                                                                                                                                                                            
 - SpecialFunctions              0.3.8                                                                                                                                                                            
116 additional packages:                                                                                                                                                                                          
 - ASTInterpreter2               0.1.1
 - AbstractFFTs                  0.2.1
 - AutoHashEquals                0.2.0
 - AxisAlgorithms                0.2.0
 - AxisArrays                    0.2.0
 - BinDeps                       0.8.6
 - Blink                         0.6.0
 - Blosc                         0.3.0
 - Calculus                      0.2.2
 - CatIndices                    0.1.0
 - CategoricalArrays             0.3.3
 - Cbc                           0.3.2
 - CodeTools                     0.4.7
 - CodecZlib                     0.4.2
 - Codecs                        0.4.0
 - ColorBrewer                   0.3.1
 - ColorTypes                    0.6.6
 - ColorVectorSpace              0.5.2
 - CommonSubexpressions          0.0.1
 - Compat                        0.49.0
 - ComputationalResources        0.2.0
 - Conda                         0.7.1
 - Contour                       0.4.0
 - CoordinateTransformations     0.4.1
 - CustomUnitRanges              0.1.0
 - DataStreams                   0.3.4
 - DataStructures                0.7.4
 - DebuggerFramework             0.1.2
 - DiffBase                      0.3.2
 - DiffEqDiffTools               0.3.0
 - DiffResults                   0.0.3
 - DiffRules                     0.0.3
 - Discretizers                  2.1.0
 - Distances                     0.5.0
 - Distributions                 0.15.0
 - DocSeeker                     0.1.0
 - DualNumbers                   0.3.0
 - FFTViews                      0.1.0
 - FFTW                          0.0.4
 - FileIO                        0.6.1
 - FixedPointNumbers             0.4.3
 - ForwardDiff                   0.7.2
 - Graphics                      0.2.0
 - HDF5                          0.8.8+             master
 - Hiccup                        0.1.1
 - HttpCommon                    0.3.0
 - HttpParser                    0.3.0
 - HttpServer                    0.2.0
 - IdentityRanges                0.1.0
 - ImageAxes                     0.4.0
 - ImageCore                     0.5.0
 - ImageFiltering                0.2.3
 - ImageMagick                   0.5.0
 - ImageMetadata                 0.4.0
 - ImageMorphology               0.0.2
 - ImageTransformations          0.4.1
 - Images                        0.12.0
 - IndirectArrays                0.3.0
 - IntervalSets                  0.1.1
 - JSON                          0.16.4
 - Juno                          0.3.2
 - LNR                           0.0.2
 - Lazy                          0.12.0
 - LegacyStrings                 0.3.0
 - LineSearches                  3.2.5
 - MacroTools                    0.4.0
 - MappedArrays                  0.0.7
 - MathProgBase                  0.7.0
 - MbedTLS                       0.5.5
 - Measures                      0.1.0
 - Media                         0.3.0
 - Missings                      0.2.5
 - Mustache                      0.3.0
 - Mux                           0.2.3
 - NLSolversBase                 4.2.0
 - NaNMath                       0.3.0
 - NamedTuples                   4.0.0
 - Nullables                     0.0.3
 - OffsetArrays                  0.4.2
 - PDMats                        0.8.0
 - PaddedViews                   0.2.0
 - Parameters                    0.8.1
 - PlotThemes                    0.2.0
 - PlotUtils                     0.4.4
 - Polynomials                   0.2.1
 - PositiveFactorizations        0.1.0
 - PyCall                        1.15.0
 - QuadGK                        0.2.0
 - RData                         0.3.0
 - RangeArrays                   0.2.0
 - Ratios                        0.2.0
 - RecipesBase                   0.2.3
 - Reexport                      0.1.0
 - Requires                      0.4.3
 - ReverseDiffSparse             0.8.1
 - Rmath                         0.3.2
 - Rotations                     0.6.1
 - SHA                           0.5.6
 - SIUnits                       0.1.0
 - ShowItLikeYouBuildIt          0.2.0
 - Showoff                       0.1.1
 - SimpleTraits                  0.5.1
 - SortingAlgorithms             0.2.0
 - StaticArrays                  0.6.6
 - StatsBase                     0.19.5
 - StatsFuns                     0.5.0
 - StringDistances               0.2.0
 - TexExtensions                 0.1.0
 - TikzPictures                  1.2.0
 - TiledIteration                0.1.0
 - Tokenize                      0.4.2
 - TranscodingStreams            0.4.1
 - URIParser                     0.3.0
 - WeakRefStrings                0.4.1
 - WebSockets                    0.4.0
 - WoodburyMatrices              0.2.2
stakaz commented 6 years ago

Hm, thats strange. I have nearly nothing special about julia. It is installed from arch repositories.

pkofod commented 6 years ago

Could it be an arch-related bug then? In Julia itself? Does the problem happen for all our tests as well?

stakaz commented 6 years ago

Yes, it is indeed, I have downloaded the official version of julia and here everything works just fine... So it has something to do how julia itself is build.

I will stick to this version for now. Thank you for help.

pkofod commented 6 years ago

Weird, but great!

cynddl commented 6 years ago

Same issue here using the latest stable version from ArchLinux. The same version and environment on Ubuntu Server work perfectly fine indeed.

pkofod commented 6 years ago

So what version of NLsolve do you have?

pkofod commented 6 years ago

Oh way, you're also on arch. This seems to be realted to the version provided by archlinux's pkg manager.

ChrisRackauckas commented 6 years ago

Who maintains that?

smldis commented 6 years ago

Have a look here https://www.archlinux.org/packages/community/x86_64/julia/

pkofod commented 6 years ago

I've contacted him via mail.

pkofod commented 6 years ago

Oh, it just occurred to me that of course he's on github :) @xyproto (sorry for the mail AND the ping)

smldis commented 6 years ago

I've filed the bug here https://bugs.archlinux.org/task/57387

eli-schwartz commented 6 years ago

According to the bug you filed, this problem is explicitly caused by Arch Linux not using the bundled LLVM. Can you confirm how you figured this cause out, and can someone please explain why julia allows building against an external LLVM if it is really true that julia simply doesn't work that way?

Because this really seems like an outrageous upstream development model, so I hope this is a bizarre misunderstanding and the problem really has nothing to do with this at all...

iamed2 commented 6 years ago

can someone please explain why julia allows building against an external LLVM if it is really true that julia simply doesn't work that way?

It is possible to have a separate version of LLVM that works with Julia. The person building Julia with their own LLVM should run the test suite to ensure that it is compatible. It's not uncommon for those working on Julia compiler internals to run or test with more than one build of LLVM.

It's also possible to specify a version of LLVM for Julia to compile with, where Julia will apply whichever patches are still necessary for that version of LLVM.

It's worth noting that all of Julia's LLVM patches that I know of are fixed in later versions of LLVM or at least have been submitted and are being reviewed. Julia does not add or change LLVM features nor does the Julia community maintain an LLVM fork.

smldis commented 6 years ago

@eli-schwartz I am not exactly sure, I think one referece can be https://github.com/JuliaLang/julia/blob/master/README.md#system-provided-libraries. And now there is https://github.com/JuliaLang/julia/issues/25905.

I guess it is not that clear about LLVM. Do you know if julia is passing the testsuite as it is packaged currently on Arch?

The only thing I can say is that all the users that came to the julia community communication channels with issues like this (I didn't see a lot of edge cases like this one thow) has been warmly welcomed and helped.

xyproto commented 6 years ago

Currently, Julia on Arch Linux is packaged with USE_SYSTEM_LLVM=1 while also depending on the llvm39 package.

The ideal situation would be to be able to depend on the latest version of LLVM instead, the llvm package.

The model Arch Linux is built around is to let package depend on each other, and always use the latest release of every package while also avoiding patches, whenever possible; to let upstream develop and packagers package. If this approach is "good" or "bad" can be discussed at length, but what's certain is that the current model of including modified LLVM sources in the Julia source code repository is not a good fit for the packaging model that Arch Linux uses.

Could Julia support the latest version of LLVM?

pkofod commented 6 years ago

Thanks for taking part in the discussion @xyproto and @eli-schwartz . To have your questions answered by the people who know and care about these things, I think it's better to move the discussion to https://github.com/JuliaLang/julia/issues/25905

nalimilan commented 6 years ago

Speaking as the maintainer of the Fedora Julia package and as a Julia developer, I don't think always using the latest LLVM version is a realistic goal. LLVM moves too fast, and even if Julia was kept up to date with LLVM changes there will always be a period of several months during which the current Julia version does not support the new LLVM which was released after it, and the next Julia version supporting it is not published yet. I had tried using the standard llvm Fedora package, but that didn't work because of this, so we ended up adding an llvm3.9 package, which is also used by other programming languages. LLVM is the only dependency for which we do this, but it's very special too because it breaks compatibility every six months.

We also ship a few patches which are useful for Julia and which are backported from more recent LLVM versions. I imagine it should be possible to avoid this now that Julia is going to stabilize, i.e. all patches could hopefully be pushed upstream in time for inclusion in the initial release or in patch releases.

xyproto commented 6 years ago

Blender and Julia are the only official packages on Arch Linux that require older versions of LLVM. (blender requires llvm35 and julia requires llvm39. I'm not counting openshadinglanguage since it can supposedly support LLVM 5 now, the package just haven't been updated yet).

If Julia can't support the latest version of LLVM, perhaps selecting a specific version of LLVM that can be used without patches could be a realistic goal?

For Arch Linux, if Julia no longer needed LLVM 3.9, it is very likely that the llvm39 package would be moved to the unofficial package repository (AUR).

nalimilan commented 6 years ago

Not requiring patches could be a realistic goal (though I cannot speak for people working on the compiler), but that's certainly too late for 1.0. So in the meantime another solution needs to be found. FWIW, note that the patches are all backported from more recent LLVM releases and they shouldn't cause problems for other users of LLVM.

eli-schwartz commented 6 years ago

As a distro policy, Arch is pretty okay with backporting fixes from upstream master when the situation warrants, so this could be a reasonable way forward. If julia only uses vanilla llvm with backported fixes, then this could be documented, and distro maintainers for julia would then know that their options are to either use the custom llvm, or ensure that the system llvm includes those fixes.

Thanks for clearing that up.

simonbyrne commented 6 years ago

I've made some suggested changes to our README here: https://github.com/JuliaLang/julia/pull/25912. Suggestions/comments welcome.