Cannot load library on Julia 1.3rc5: MethodError: no method matching iterate(::Nothing)

[chriselrod]

On Julia 1.3rc5:

julia> using VML
[ Info: Precompiling VML [c8ce9da6-5d36-5c03-b118-5a70151be7bc]
ERROR: LoadError: LoadError: MethodError: no method matching iterate(::Nothing)
Closest candidates are:
  iterate(::Core.SimpleVector) at essentials.jl:600
  iterate(::Core.SimpleVector, ::Any) at essentials.jl:600
  iterate(::ExponentialBackOff) at error.jl:218
  ...
Stacktrace:
 [1] isempty(::Nothing) at ./essentials.jl:736
 [2] __init__() at /home/chriselrod/.julia/packages/VML/lA63S/src/setup.jl:5
 [3] top-level scope at /home/chriselrod/.julia/packages/VML/lA63S/src/setup.jl:18
 [4] include at ./boot.jl:328 [inlined]
 [5] include_relative(::Module, ::String) at ./loading.jl:1105
 [6] include at ./Base.jl:31 [inlined]
 [7] include(::String) at /home/chriselrod/.julia/packages/VML/lA63S/src/VML.jl:3
 [8] top-level scope at /home/chriselrod/.julia/packages/VML/lA63S/src/VML.jl:10
 [9] include at ./boot.jl:328 [inlined]
 [10] include_relative(::Module, ::String) at ./loading.jl:1105
 [11] include(::Module, ::String) at ./Base.jl:31
 [12] top-level scope at none:2
 [13] eval at ./boot.jl:330 [inlined]
 [14] eval(::Expr) at ./client.jl:425
 [15] top-level scope at ./none:3
in expression starting at /home/chriselrod/.julia/packages/VML/lA63S/src/setup.jl:18
in expression starting at /home/chriselrod/.julia/packages/VML/lA63S/src/VML.jl:10
ERROR: Failed to precompile VML [c8ce9da6-5d36-5c03-b118-5a70151be7bc] to /home/chriselrod/.julia/compiled/v1.3/VML/WlHoi_R5WJb.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1283
 [3] _require(::Base.PkgId) at ./loading.jl:1024
 [4] require(::Base.PkgId) at ./loading.jl:922
 [5] require(::Module, ::Symbol) at ./loading.jl:917

[aminya]

What OS are you using? Have you installed MKL?

[chriselrod]

Oops -- I saw a weird error not mentioning libraries, and didn't investigate further in my rush to write a post. I do not have MKL.jl installed, having built with MKL from source

julia> using LinearAlgebra

julia> BLAS.vendor()
:mkl

It looks like I should be getting a different error. The README claims that

Alternatively you can install MKL directly from intel.

but from the setup script, it looks like this is not actually supported?

[aminya]

You need to either download MKL and put it on the path, or use the MKL.jl. Both work.

[chriselrod]

> echo $LD_LIBRARY_PATH | grep linux/mkl/lib/intel64
/home/chriselrod/intel/compilers_and_libraries_2019.3.199/linux/compiler/lib/intel64_lin /home/chriselrod/intel/compilers_and_libraries_2019.3.199/linux/compiler/lib/intel64_lin /home/chriselrod/intel/compilers_and_libraries_2019.3.199/linux/mkl/lib/intel64_lin /opt/intel/compilers_and_libraries_2019.4.243/linux/compiler/lib/intel64_lin /opt/intel/compilers_and_libraries_2019.4.243/linux/mpi/intel64/libfabric/lib /opt/intel/compilers_and_libraries_2019.4.243/linux/mpi/intel64/lib/release /opt/intel/compilers_and_libraries_2019.4.243/linux/mpi/intel64/lib /opt/intel/compilers_and_libraries_2019.4.243/linux/ipp/lib/intel64 /opt/intel/compilers_and_libraries_2019.4.243/linux/compiler/lib/intel64_lin /opt/intel/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64_lin /opt/intel/compilers_and_libraries_2019.4.243/linux/tbb/lib/intel64/gcc4.7 /opt/intel/compilers_and_libraries_2019.4.243/linux/tbb/lib/intel64/gcc4.7 /opt/intel/compilers_and_libraries_2019.4.243/linux/daal/lib/intel64_lin /opt/intel/compilers_and_libraries_2019.4.243/linux/daal/../tbb/lib/intel64_lin/gcc4.4

Notice the third path listed. Looking inside:

> ls /home/chriselrod/intel/compilers_and_libraries_2019.3.199/linux/mkl/lib/intel64_lin
libmkl_avx2.so*        libmkl_avx.so*         libmkl_core.a    libmkl_gf_ilp64.a    libmkl_gf_lp64.so*     libmkl_intel_ilp64.a    libmkl_intel_lp64.so*    libmkl_lapack95_ilp64.a  libmkl_mc.so*        libmkl_sequential.so*  libmkl_vml_avx2.so*        libmkl_vml_avx.so*   libmkl_vml_mc2.so*  locale/
libmkl_avx512_mic.so*  libmkl_blas95_ilp64.a  libmkl_core.so*  libmkl_gf_ilp64.so*  libmkl_gnu_thread.a    libmkl_intel_ilp64.so*  libmkl_intel_thread.a    libmkl_lapack95_lp64.a   libmkl_rt.so*        libmkl_tbb_thread.a    libmkl_vml_avx512_mic.so*  libmkl_vml_cmpt.so*  libmkl_vml_mc3.so*
libmkl_avx512.so*      libmkl_blas95_lp64.a   libmkl_def.so*   libmkl_gf_lp64.a     libmkl_gnu_thread.so*  libmkl_intel_lp64.a     libmkl_intel_thread.so*  libmkl_mc3.so*           libmkl_sequential.a  libmkl_tbb_thread.so*  libmkl_vml_avx512.so*      libmkl_vml_def.so*   libmkl_vml_mc.so*

Or, to be more succinct:

> ls /home/chriselrod/intel/compilers_and_libraries_2019.3.199/linux/mkl/lib/intel64_lin | grep vml
libmkl_vml_avx2.so
libmkl_vml_avx512_mic.so
libmkl_vml_avx512.so
libmkl_vml_avx.so
libmkl_vml_cmpt.so
libmkl_vml_def.so
libmkl_vml_mc2.so
libmkl_vml_mc3.so
libmkl_vml_mc.so

I already have VML on my load path, so I don't think I should have to install anything else?

[aminya]

@Crown421 has a Linux system. So he can help you better.

[Crown421]

I had the same error on one of my systems (iterate over Nothing), I thought I had fixed it with 009483696e0fe7645d752ac07b49f9b704446460

On a fresh julia REPL, can you run and see if it gives you something back?

julia> using Libdl
julia> Libdl.dlpath(:libmkl_rt)

Also, can you check your local setup.jl file / update the package? Because with the aforementioned commit there should be nothing to iterate over in the __init__.

[chriselrod]

julia> using Libdl

julia> Libdl.dlpath(:libmkl_rt)
"/home/chriselrod/intel/compilers_and_libraries_2019.3.199/linux/mkl/lib/intel64_lin/libmkl_rt.so"

I am currently on VML master. I still get the same error.

julia> using VML
[ Info: Precompiling VML [c8ce9da6-5d36-5c03-b118-5a70151be7bc]
ERROR: LoadError: LoadError: MethodError: no method matching iterate(::Nothing)
Closest candidates are:
  iterate(::Core.SimpleVector) at essentials.jl:602
  iterate(::Core.SimpleVector, ::Any) at essentials.jl:602
  iterate(::ExponentialBackOff) at error.jl:253
  ...
Stacktrace:
 [1] isempty(::Nothing) at ./essentials.jl:738
 [2] __init__() at /home/chriselrod/.julia/packages/VML/SaEFf/src/setup.jl:5
 [3] top-level scope at /home/chriselrod/.julia/packages/VML/SaEFf/src/setup.jl:18
 [4] include(::Module, ::String) at ./Base.jl:377
 [5] include(::String) at /home/chriselrod/.julia/packages/VML/SaEFf/src/VML.jl:3
 [6] top-level scope at /home/chriselrod/.julia/packages/VML/SaEFf/src/VML.jl:10
 [7] include(::Module, ::String) at ./Base.jl:377
 [8] top-level scope at none:2
 [9] eval at ./boot.jl:331 [inlined]
 [10] eval(::Expr) at ./client.jl:449
 [11] top-level scope at ./none:3
in expression starting at /home/chriselrod/.julia/packages/VML/SaEFf/src/setup.jl:18
in expression starting at /home/chriselrod/.julia/packages/VML/SaEFf/src/VML.jl:10
ERROR: Failed to precompile VML [c8ce9da6-5d36-5c03-b118-5a70151be7bc] to /home/chriselrod/.julia/compiled/v1.4/VML/WlHoi_8nmxv.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1263
 [3] _require(::Base.PkgId) at ./loading.jl:1022
 [4] require(::Base.PkgId) at ./loading.jl:920
 [5] require(::Module, ::Symbol) at ./loading.jl:915

Checking the contents of SaEFf/src/setup.jl

function __init__()
    MKLpkgid = Base.PkgId(Base.UUID("33e6dc65-8f57-5167-99aa-e5a354878fb2"), "MKL")
    mklpath = Base.locate_package(MKLpkgid)
    if !isempty(mklpath)
        libpath = normpath(joinpath(dirname(mklpath), "../deps/usr/lib"))
        push!(Libdl.DL_LOAD_PATH, libpath)
    elseif isempty(Libdl.find_library(rtlib))
        error("Could not find MKL shared libraries. Please add MKL.jl or install MKL via the intel website. See the github repository for more details.)")
    end

    Libdl.dlopen(rtlib, RTLD_GLOBAL)
    Libdl.dlopen(corelib, RTLD_GLOBAL) # maybe only needed on mac
    Libdl.dlopen(lib, RTLD_GLOBAL)

end

The error comes from:

julia> isempty(nothing)
ERROR: MethodError: no method matching iterate(::Nothing)
Closest candidates are:
  iterate(::Core.SimpleVector) at essentials.jl:602
  iterate(::Core.SimpleVector, ::Any) at essentials.jl:602
  iterate(::ExponentialBackOff) at error.jl:253
  ...
Stacktrace:
 [1] isempty(::Nothing) at ./essentials.jl:738
 [2] top-level scope at REPL[31]:1

This happens because I do not have MKL.jl installed:

julia> MKLpkgid = Base.PkgId(Base.UUID("33e6dc65-8f57-5167-99aa-e5a354878fb2"), "MKL")
MKL [33e6dc65-8f57-5167-99aa-e5a354878fb2]

julia> mklpath = Base.locate_package(MKLpkgid)

julia> mklpath === nothing
true

I think what you want to do is a check vs nothing, not a isempty.

[Crown421]

Very interesting, I thought the machine I tested this on also does not have MKL.jl, but instead uses MKL_slim from the AUR. Well, testing on WSL Ubuntu confirms that you are right, I just changed it. Writing something for Travis to check this against apt installed MKL is on my list for the weekend.

[chriselrod]

Very interesting, I thought the machine I tested this on also does not have MKL.jl, but instead uses MKL_slim from the AUR. Well, testing on WSL Ubuntu confirms that you are right, I just changed it. Writing something for Travis to check this against apt installed MKL is on my list for the weekend.

Thanks, I just tested and confirmed that it now works locally!