LAPACK error arrising from PyPlot

[PhillipBC]

Hi,

This problem first arose using Julia 1.4.2, and I then changed to version 1.5.1 and the problem was still there.

If I create a matrix, then get its eigenvalues, then plot a histogram of the eigenvalues, then get the eigenvalues again, I get the error:

** On entry to DGEBAL parameter number  3 had an illegal value
 ** On entry to DGEHRD  parameter number  2 had an illegal value
 ** On entry to DHSEQR parameter number  4 had an illegal value
ArgumentError: invalid argument #4 to LAPACK call
Stacktrace:
 [1] chklapackerror at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\LinearAlgebra\src\lapack.jl:36 [inlined]
 [2] geevx!(::Char, ::Char, ::Char, ::Char, ::Array{Float64,2}) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\LinearAlgebra\src\lapack.jl:2082
 [3] eigvals!(::Array{Float64,2}; permute::Bool, scale::Bool, sortby::typeof(LinearAlgebra.eigsortby)) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\LinearAlgebra\src\eigen.jl:293
 [4] eigvals! at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\LinearAlgebra\src\eigen.jl:292 [inlined]
 [5] #eigvals#73 at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\LinearAlgebra\src\eigen.jl:326 [inlined]
 [6] eigvals(::Array{Float64,2}) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\LinearAlgebra\src\eigen.jl:326
 [7] top-level scope at F:\MEGA\Julia_Code\00_Temperature\errorfind.jl:10

Below is a minimum working example that causes the error:

using LinearAlgebra
using PyCall
using PyPlot

D = 1000
M = randn(D,D)
ei = real(eigvals(M))
PyPlot.hist(ei)

Then call the eigensolver again

ei = real(eigvals(M))

Gives:

** On entry to DGEBAL parameter number  3 had an illegal value
 ** On entry to DGEHRD  parameter number  2 had an illegal value
 ** On entry to DHSEQR parameter number  4 had an illegal value
ERROR: ArgumentError: invalid argument #4 to LAPACK call
Stacktrace:
 [1] chklapackerror at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\LinearAlgebra\src\lapack.jl:36 [inlined]
 [2] geevx!(::Char, ::Char, ::Char, ::Char, ::Array{Float64,2}) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\LinearAlgebra\src\lapack.jl:2082
 [3] eigvals!(::Array{Float64,2}; permute::Bool, scale::Bool, sortby::typeof(LinearAlgebra.eigsortby)) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\LinearAlgebra\src\eigen.jl:293
 [4] eigvals! at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\LinearAlgebra\src\eigen.jl:292 [inlined]
 [5] #eigvals#73 at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\LinearAlgebra\src\eigen.jl:326 [inlined]
 [6] eigvals(::Array{Float64,2}) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\LinearAlgebra\src\eigen.jl:326
 [7] top-level scope at none:1

I originally posted about the error on Julia discourse: ArgumentError: invalid argument #4 to LAPACK call back in September.

As you will see there, one can also reproduce the error using Plots.jl with the pyplot() backend.

Version Info:

Julia Version 1.5.1
Commit 697e782ab8 (2020-08-25 20:08 UTC)       
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake)

[stevengj]

Sounds similar to https://github.com/JuliaPy/PyCall.jl/issues/443 … that is, PyPlot loads matplotlib which loads numpy, and numpy loads LAPACK and BLAS. It's possible that these are somehow conflicting with the LAPACK and BLAS libraries used by Julia.

Are you using the MKL with Julia, or just the default linear algebra (OpenBLAS)?

[PhillipBC]

Sounds similar to JuliaPy/PyCall.jl#443 … that is, PyPlot loads matplotlib which loads numpy, and numpy loads LAPACK and BLAS. It's possible that these are somehow conflicting with the LAPACK and BLAS libraries used by Julia.

Are you using the MKL with Julia, or just the default linear algebra (OpenBLAS)?

I am using the default one. I tried to use MKL but I kept getting a permissions error when building the MKL package.

[stevengj]

Does the same problem occur if you do:

using PyCall
pyimport("numpy")

?

[PhillipBC]

If I do np = pyimport("numpy") and then use np.linalg.eigvals(M) The error does not occur.

[stevengj]

But does the error then occur with the Julia eigvals function?

[PhillipBC]

Yes it does still occur when using the Julia eigvals function.

[stevengj]

So the problem is not PyPlot (or PyCall), it is a shared-library conflict when loading numpy.

That is not supposed to happen with Julia's default OpenBLAS library, because all of the symbols get a 64_ suffix on 64-bit systems. Or are using a 32-bit build of Julia (e.g. on Windows)?

[PhillipBC]

I am running a 64-bit version on Windows.

[stevengj]

In that case I would file a Julia issue (since it's not related to the PyPlot package per se). It shouldn't be possible for importing numpy to break LinearAlgebra

[PhillipBC]

I think I was confused by what you meant. I thought you meant add pyimport(numpy) to the minimum example above and see if the error still occurs (which it does).

Did you mean just try

using LinearAlgebra
using PyCall
pyimport(numpy)

D = 1000
M = randn(D,D)
ei = real(eigvals(M))

This does not cause any error. The error only arises from a PyPlot call followed by an eigvals call.

[jamblejoe]

I can reproduce the following on Windows 10, Julia 1.5.3, openblas,

using LinearAlgebra
using PyPlot
eigvals(rand(10,10))
hist(rand(10))
eigvals(rand(10,10))

gives

eigvals(rand(10,10))
 ** On entry to DGEBAL parameter number  3 had an illegal value
 ** On entry to DGEHRD  parameter number  2 had an illegal value
 ** On entry to DHSEQR parameter number  4 had an illegal value
ERROR: ArgumentError: invalid argument #4 to LAPACK call
Stacktrace:
 [1] chklapackerror at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\LinearAlgebra\src\lapack.jl:36
 [inlined]
 [2] geevx!(::Char, ::Char, ::Char, ::Char, ::Array{Float64,2}) at C:\buildbot\worker\package_win64\build\usr\share\juli
a\stdlib\v1.5\LinearAlgebra\src\lapack.jl:2082
 [3] eigvals!(::Array{Float64,2}; permute::Bool, scale::Bool, sortby::typeof(LinearAlgebra.eigsortby)) at C:\buildbot\wo
rker\package_win64\build\usr\share\julia\stdlib\v1.5\LinearAlgebra\src\eigen.jl:293
 [4] eigvals! at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\LinearAlgebra\src\eigen.jl:292 [inli
ned]
 [5] #eigvals#73 at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\LinearAlgebra\src\eigen.jl:326 [i
nlined]
 [6] eigvals(::Array{Float64,2}) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\LinearAlgebra\src
\eigen.jl:326
 [7] top-level scope at REPL[5]:1

while

using LinearAlgebra
using PyPlot
eigvals(rand(10,10))
plot(rand(10))
eigvals(rand(10,10))

works fine and

using LinearAlgebra
using PyCall
np = pyimport("numpy")

eigvals(rand(10,10))
np.linalg.eigvals(rand(10,10))

works fine, too.

julia> versioninfo()
Julia Version 1.5.3
Commit 788b2c77c1 (2020-11-09 13:37 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i5-4590 CPU @ 3.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, haswell)
Environment:
  JULIA_NUM_THREADS = 4

Packages:

  [c52e3926] Atom v0.12.24
  [052768ef] CUDA v1.2.1
  [634d3b9d] DrWatson v1.14.7
  [7073ff75] IJulia v1.21.2
  [e5e0dc1b] Juno v0.8.4
  [91a5bcdd] Plots v1.8.1
  [438e738f] PyCall v1.92.1
  [d330b81b] PyPlot v2.9.0

[stevengj]

It's still a Julia problem, not a PyPlot problem. hist is apparently calling some numpy function that affects Julia's LAPACK/OpenBLAS, which shouldn't be possible.