JuliaLinearAlgebra / MKL.jl

Intel MKL linear algebra backend for Julia
Other
206 stars 32 forks source link

MKL isn't built on Windows #24

Closed aminya closed 4 years ago

aminya commented 4 years ago

CI on my V0.1 branch: https://travis-ci.com/aminya/MKL.jl/builds/142809478 https://ci.appveyor.com/project/andreasnoack/mkl-jl/builds/29837534

Travis on my patch-3 branch (before REPL-Startup fix) https://travis-ci.com/aminya/MKL.jl/builds/142824221

This shows that the problem is something external like PackageCompiler because I didn't have any issue building patch-3 at that time.

Local build spoiler: ```julia (v1.3) pkg> add https://github.com/JuliaComputing/MKL.jl Updating git-repo `https://github.com/JuliaComputing/MKL.jl` Updating git-repo `https://github.com/JuliaComputing/MKL.jl` Resolving package versions... Updating `C:\Users\yahyaaba\.julia\environments\v1.3\Project.toml` [33e6dc65] + MKL v0.1.0 #master (https://github.com/JuliaComputing/MKL.jl) Updating `C:\Users\yahyaaba\.julia\environments\v1.3\Manifest.toml` [85c772de] + AbstractNumbers v0.2.0 [537997a7] ↑ AbstractPlotting v0.9.10 ⇒ v0.9.16 [39de3d68] - AxisArrays v0.3.3 [e1450e63] + BufferedStreams v1.0.0 [aafaddc9] - CatIndices v0.2.0 [35d6a980] + ColorSchemes v3.5.0 [ed09eef8] - ComputationalResources v0.3.0 [150eb455] - CoordinateTransformations v0.5.0 [dc8bdbbb] - CustomUnitRanges v0.2.0 [2e619515] + Expat_jll v2.2.7+0 [4f61f5a4] - FFTViews v0.3.0 [e9467ef8] ↑ GLMakie v0.0.8 ⇒ v0.0.12 [0862f596] + HTTPClient v0.2.1 [bbac6d45] - IdentityRanges v0.3.0 [2803e5a7] - ImageAxes v0.6.2 [6a3955dd] - ImageFiltering v0.6.9 [02fcd773] - ImageTransformations v0.8.1 [9b13fd28] - IndirectArrays v0.5.1 [c8e1da08] - IterTools v1.3.0 [b27032c2] + LibCURL v0.5.2 [522f3ed2] + LibExpat v0.6.0 [2ec943e9] + Libz v1.0.1 [33e6dc65] + MKL v0.1.0 #master (https://github.com/JuliaComputing/MKL.jl) [9b87118b] + PackageCompiler v0.6.5 [b3c3ace0] - RangeArrays v0.3.2 [6038ab10] - Rotations v0.13.0 [699a6c99] - SimpleTraits v0.9.1 [06e1c1a7] - TiledIteration v0.2.3 [c17dfb99] + WinRPM v0.4.3 (v1.3) pkg> build MKL Building LibCURL ────────→ `C:\Users\yahyaaba\.julia\packages\LibCURL\lWJxD\deps\build.log` Building WinRPM ─────────→ `C:\Users\yahyaaba\.julia\packages\WinRPM\BfpQI\deps\build.log` Building PackageCompiler → `C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\deps\build.log` Building MKL ────────────→ `C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\deps\build.log` ┌ Error: Error building `MKL`, showing the last 100 of log: │ CRC32c ───────── 0.009568 seconds │ SHA ──────────── 0.175667 seconds │ FileWatching ─── 0.087281 seconds │ Unicode ──────── 0.010049 seconds │ Mmap ─────────── 0.072874 seconds │ Serialization ── 0.880602 seconds │ Libdl ────────── 0.028785 seconds │ Markdown ─────── 1.004954 seconds │ LibGit2 ──────── 2.635718 seconds │ Logging ──────── 0.436265 seconds │ Sockets ──────── 1.678002 seconds │ Printf ───────── 0.013321 seconds │ Profile ──────── 0.223180 seconds │ Dates ────────── 2.114574 seconds │ DelimitedFiles ─ 0.115828 seconds │ Random ───────── 0.495196 seconds │ UUIDs ────────── 0.019967 seconds │ Future ───────── 0.011228 seconds │ LinearAlgebra ── 10.850114 seconds │ SparseArrays ─── 4.239812 seconds │ SuiteSparse ──── 1.437989 seconds │ Distributed ──── 4.812321 seconds │ SharedArrays ─── 0.163400 seconds │ Pkg ──────────── 12.755205 seconds │ Test ─────────── 0.912637 seconds │ REPL ─────────── 0.853262 seconds │ Statistics ───── 0.160607 seconds │ Stdlibs total ── 49.844954 seconds │ Sysimage built. Summary: │ Total ─────── 74.055858 seconds │ Base: ─────── 24.207745 seconds 32.6885% │ Stdlibs: ──── 49.844954 seconds 67.3072% │ c:/users/yahyaaba/.julia/packages/winrpm/bfpqi/deps/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/8.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: :0: syntax error │ Warning: .drectve `-export:ccalllib_C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\deps\usr\bin\mkl_rt.dll,data ' unrecognized │ c:/users/yahyaaba/.julia/packages/winrpm/bfpqi/deps/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/8.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot export MKL: symbol not found │ c:/users/yahyaaba/.julia/packages/winrpm/bfpqi/deps/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/8.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot export Users: symbol not found │ c:/users/yahyaaba/.julia/packages/winrpm/bfpqi/deps/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/8.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot export ccalllib_C:: symbol not found │ c:/users/yahyaaba/.julia/packages/winrpm/bfpqi/deps/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/8.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot export packages: symbol not found │ collect2.exe: error: ld returned 1 exit status │ ┌ Warning: On Windows, creating file symlinks requires Administrator privileges │ └ @ Base.Filesystem file.jl:848 │ [ Info: Replacing libblas_name in C:\Julia-1.3.1-MKL\bin\..\share\julia\base\build_h.jl │ [ Info: Checking if we need to update PATH... │ [ Info: Checking sysimg.jl │ [ Info: Could not find init function in sysimg.jl │ [ Info: Checking Base.jl │ [ Info: Successfully modified Base.jl │ ┌ Info: Building `inference.o`: │ └ `'C:\Julia-1.3.1-MKL\bin\julia.exe' -C native --output-ji 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\basecompiler.ji' --output-o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\basecompiler.o' compiler/compiler.jl` │ ┌ Info: Building `sys.o`: │ └ `'C:\Julia-1.3.1-MKL\bin\julia.exe' -C native --output-ji 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.ji' --output-o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.o' -J 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\basecompiler.ji' --startup-file=no sysimg.jl` │ ERROR: LoadError: failed process: Process(setenv(`'C:\Users\yahyaaba\.julia\packages\WinRPM\BfpQI\deps\usr\x86_64-w64-mingw32\sys-root\mingw\bin\gcc.exe' --sysroot 'C:\Users\yahyaaba\.julia\packages\WinRPM\BfpQI\deps\usr\x86_64-w64-mingw32\sys-root' -shared '-DJULIAC_PROGRAM_LIBNAME="C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.dll"' -o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.dll' -Wl,--whole-archive 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.o' -Wl,--no-whole-archive -std=gnu99 '-IC:\Julia-1.3.1-MKL\include\julia' -DJULIA_ENABLE_THREADING=1 '-LC:\Julia-1.3.1-MKL\bin' -Wl,--stack,8388608 -ljulia -lopenlibm -m64 -Wl,--export-all-symbols`,["PATH=C:\\Users\\yahyaaba\\.julia\\packages\\WinRPM\\BfpQI\\deps\\usr\\x86_64-w64-mingw32\\sys-root\\mingw\\bin;"]), ProcessExited(1)) [1] │ │ Stacktrace: │ [1] pipeline_error at .\process.jl:525 [inlined] │ [2] #run#565(::Bool, ::typeof(run), ::Cmd) at .\process.jl:440 │ [3] run at .\process.jl:438 [inlined] │ [4] run_PATH(::Cmd) at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\static_julia.jl:23 │ [5] (::PackageCompiler.var"#16#18")() at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\static_julia.jl:309 │ [6] cd(::PackageCompiler.var"#16#18", ::String) at .\file.jl:93 │ [7] build_shared(::String, ::String, ::Bool, ::String, ::Bool, ::Nothing, ::Nothing, ::Cmd, ::Nothing) at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\static_julia.jl:308 │ [8] (::PackageCompiler.var"#40#41"{Bool,String})() at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\system_image.jl:46 │ [9] cd(::PackageCompiler.var"#40#41"{Bool,String}, ::String) at .\file.jl:93 │ [10] #compile_system_image#39(::Bool, ::typeof(PackageCompiler.compile_system_image), ::String, ::String) at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\system_image.jl:15 │ [11] #compile_system_image at .\none:0 [inlined] │ [12] get_backup!(::Bool, ::String) at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\system_image.jl:63 │ [13] force_native_image! at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\api.jl:123 [inlined] (repeats 2 times) │ [14] enable_mkl_startup(::String) at C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\src\install.jl:138 │ [15] top-level scope at C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\deps\build.jl:9 │ [16] include at .\boot.jl:328 [inlined] │ [17] include_relative(::Module, ::String) at .\loading.jl:1105 │ [18] include(::Module, ::String) at .\Base.jl:31 │ [19] include(::String) at .\client.jl:424 │ [20] top-level scope at none:5 │ in expression starting at C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\deps\build.jl:9 │ caused by [exception 1] │ IOError: symlink: operation not permitted (EPERM) │ Stacktrace: │ [1] uv_error at .\libuv.jl:97 [inlined] │ [2] symlink(::String, ::String) at .\file.jl:851 │ [3] probe_symlink_creation(::String) at C:\Users\yahyaaba\.julia\packages\BinaryProvider\kcGxO\src\PlatformEngines.jl:121 │ [4] #probe_platform_engines!#30(::Bool, ::typeof(BinaryProvider.probe_platform_engines!)) at C:\Users\yahyaaba\.julia\packages\BinaryProvider\kcGxO\src\PlatformEngines.jl:175 │ [5] probe_platform_engines! at C:\Users\yahyaaba\.julia\packages\BinaryProvider\kcGxO\src\PlatformEngines.jl:169 [inlined] │ [6] __init__() at C:\Users\yahyaaba\.julia\packages\BinaryProvider\kcGxO\src\BinaryProvider.jl:28 │ [7] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:692 │ [8] _require_search_from_serialized(::Base.PkgId, ::String) at .\loading.jl:776 │ [9] _require(::Base.PkgId) at .\loading.jl:1001 │ [10] require(::Base.PkgId) at .\loading.jl:922 │ [11] require(::Module, ::Symbol) at .\loading.jl:917 │ [12] include at .\boot.jl:328 [inlined] │ [13] include_relative(::Module, ::String) at .\loading.jl:1105 │ [14] include(::Module, ::String) at .\Base.jl:31 │ [15] include(::String) at .\client.jl:424 │ [16] top-level scope at C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\deps\build.jl:4 │ [17] include at .\boot.jl:328 [inlined] │ [18] include_relative(::Module, ::String) at .\loading.jl:1105 │ [19] include(::Module, ::String) at .\Base.jl:31 │ [20] include(::String) at .\client.jl:424 │ [21] top-level scope at none:5 │ Build shared library "C:\\Users\\yahyaaba\\.julia\\packages\\PackageCompiler\\4yNnV\\sysimg\\backup\\native\\sys.dll": │ `'C:\Users\yahyaaba\.julia\packages\WinRPM\BfpQI\deps\usr\x86_64-w64-mingw32\sys-root\mingw\bin\gcc.exe' --sysroot 'C:\Users\yahyaaba\.julia\packages\WinRPM\BfpQI\deps\usr\x86_64-w64-mingw32\sys-root' -shared '-DJULIAC_PROGRAM_LIBNAME="C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.dll"' -o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.dll' -Wl,--whole-archive 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.o' -Wl,--no-whole-archive -std=gnu99 '-IC:\Julia-1.3.1-MKL\include\julia' -DJULIA_ENABLE_THREADING=1 '-LC:\Julia-1.3.1-MKL\bin' -Wl,--stack,8388608 -ljulia -lopenlibm -m64 -Wl,--export-all-symbols` │ │ Full log at C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\deps\build.log └ @ Pkg.Operations D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Pkg\src\backwards_compatible_isolation.jl:649 ``` **Full Log** ```julia essentials.jl ctypes.jl generator.jl reflection.jl options.jl promotion.jl tuple.jl pair.jl traits.jl range.jl expr.jl error.jl bool.jl number.jl int.jl operators.jl pointer.jl refvalue.jl indices.jl array.jl abstractarray.jl bitarray.jl bitset.jl abstractdict.jl abstractset.jl iterators.jl namedtuple.jl docs/core.jl ordering.jl sort.jl compiler/utilities.jl compiler/validation.jl compiler/inferenceresult.jl compiler/params.jl compiler/inferencestate.jl compiler/typeutils.jl compiler/typelimits.jl compiler/typelattice.jl compiler/tfuncs.jl compiler/abstractinterpretation.jl compiler/typeinfer.jl compiler/optimize.jl compiler/ssair/driver.jl compiler/ssair/ir.jl compiler/ssair/domtree.jl compiler/ssair/slot2ssa.jl compiler/ssair/queries.jl compiler/ssair/passes.jl compiler/ssair/inlining.jl compiler/ssair/verify.jl compiler/ssair/legacy.jl compiler/bootstrap.jl coreio.jl exports.jl essentials.jl ctypes.jl gcutils.jl generator.jl reflection.jl options.jl promotion.jl tuple.jl expr.jl pair.jl traits.jl range.jl error.jl bool.jl number.jl int.jl operators.jl pointer.jl refvalue.jl refpointer.jl checked.jl indices.jl array.jl abstractarray.jl subarray.jl views.jl baseext.jl ntuple.jl abstractdict.jl iterators.jl namedtuple.jl hashing.jl rounding.jl float.jl twiceprecision.jl complex.jl rational.jl multinverses.jl abstractarraymath.jl arraymath.jl simdloop.jl reduce.jl reshapedarray.jl reinterpretarray.jl bitarray.jl bitset.jl multimedia.jl some.jl dict.jl abstractset.jl set.jl char.jl strings/basic.jl strings/string.jl strings/substring.jl build_h.jl version_git.jl osutils.jl c.jl io.jl iobuffer.jl intfuncs.jl strings/strings.jl strings/search.jl strings/unicode.jl strings/util.jl strings/io.jl parse.jl shell.jl regex.jl pcre.jl show.jl arrayshow.jl methodshow.jl cartesian.jl multidimensional.jl permuteddimsarray.jl broadcast.jl missing.jl version.jl sysinfo.jl libc.jl env.jl linked_list.jl condition.jl threads.jl lock.jl task.jl weakkeydict.jl logging.jl libuv.jl uv_constants.jl asyncevent.jl iostream.jl stream.jl filesystem.jl cmd.jl process.jl grisu/grisu.jl secretbuffer.jl floatfuncs.jl math.jl reducedim.jl accumulate.jl ordering.jl sort.jl fastmath.jl Enums.jl gmp.jl mpfr.jl combinatorics.jl hashing2.jl irrationals.jl mathconstants.jl printf.jl meta.jl channels.jl deepcopy.jl download.jl summarysize.jl errorshow.jl stacktraces.jl initdefs.jl threadcall.jl uuid.jl loading.jl util.jl asyncmap.jl experimental.jl deprecated.jl docs\basedocs.jl client.jl docs\Docs.jl Base ─────────── 24.207745 seconds Base64 ───────── 3.632059 seconds CRC32c ───────── 0.009568 seconds SHA ──────────── 0.175667 seconds FileWatching ─── 0.087281 seconds Unicode ──────── 0.010049 seconds Mmap ─────────── 0.072874 seconds Serialization ── 0.880602 seconds Libdl ────────── 0.028785 seconds Markdown ─────── 1.004954 seconds LibGit2 ──────── 2.635718 seconds Logging ──────── 0.436265 seconds Sockets ──────── 1.678002 seconds Printf ───────── 0.013321 seconds Profile ──────── 0.223180 seconds Dates ────────── 2.114574 seconds DelimitedFiles ─ 0.115828 seconds Random ───────── 0.495196 seconds UUIDs ────────── 0.019967 seconds Future ───────── 0.011228 seconds LinearAlgebra ── 10.850114 seconds SparseArrays ─── 4.239812 seconds SuiteSparse ──── 1.437989 seconds Distributed ──── 4.812321 seconds SharedArrays ─── 0.163400 seconds Pkg ──────────── 12.755205 seconds Test ─────────── 0.912637 seconds REPL ─────────── 0.853262 seconds Statistics ───── 0.160607 seconds Stdlibs total ── 49.844954 seconds Sysimage built. Summary: Total ─────── 74.055858 seconds Base: ─────── 24.207745 seconds 32.6885% Stdlibs: ──── 49.844954 seconds 67.3072% c:/users/yahyaaba/.julia/packages/winrpm/bfpqi/deps/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/8.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: :0: syntax error Warning: .drectve `-export:ccalllib_C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\deps\usr\bin\mkl_rt.dll,data ' unrecognized c:/users/yahyaaba/.julia/packages/winrpm/bfpqi/deps/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/8.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot export MKL: symbol not found c:/users/yahyaaba/.julia/packages/winrpm/bfpqi/deps/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/8.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot export Users: symbol not found c:/users/yahyaaba/.julia/packages/winrpm/bfpqi/deps/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/8.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot export ccalllib_C:: symbol not found c:/users/yahyaaba/.julia/packages/winrpm/bfpqi/deps/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/8.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot export packages: symbol not found collect2.exe: error: ld returned 1 exit status ┌ Warning: On Windows, creating file symlinks requires Administrator privileges └ @ Base.Filesystem file.jl:848 [ Info: Replacing libblas_name in C:\Julia-1.3.1-MKL\bin\..\share\julia\base\build_h.jl [ Info: Checking if we need to update PATH... [ Info: Checking sysimg.jl [ Info: Could not find init function in sysimg.jl [ Info: Checking Base.jl [ Info: Successfully modified Base.jl ┌ Info: Building `inference.o`: └ `'C:\Julia-1.3.1-MKL\bin\julia.exe' -C native --output-ji 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\basecompiler.ji' --output-o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\basecompiler.o' compiler/compiler.jl` ┌ Info: Building `sys.o`: └ `'C:\Julia-1.3.1-MKL\bin\julia.exe' -C native --output-ji 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.ji' --output-o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.o' -J 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\basecompiler.ji' --startup-file=no sysimg.jl` ERROR: LoadError: failed process: Process(setenv(`'C:\Users\yahyaaba\.julia\packages\WinRPM\BfpQI\deps\usr\x86_64-w64-mingw32\sys-root\mingw\bin\gcc.exe' --sysroot 'C:\Users\yahyaaba\.julia\packages\WinRPM\BfpQI\deps\usr\x86_64-w64-mingw32\sys-root' -shared '-DJULIAC_PROGRAM_LIBNAME="C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.dll"' -o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.dll' -Wl,--whole-archive 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.o' -Wl,--no-whole-archive -std=gnu99 '-IC:\Julia-1.3.1-MKL\include\julia' -DJULIA_ENABLE_THREADING=1 '-LC:\Julia-1.3.1-MKL\bin' -Wl,--stack,8388608 -ljulia -lopenlibm -m64 -Wl,--export-all-symbols`,["PATH=C:\\Users\\yahyaaba\\.julia\\packages\\WinRPM\\BfpQI\\deps\\usr\\x86_64-w64-mingw32\\sys-root\\mingw\\bin;C:\\WINDOWS\\system32\\WBEM;C:\\Program Files (x86)\\Common Files\\Oracle\\Java\\javapath;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;"]), ProcessExited(1)) [1] Stacktrace: [1] pipeline_error at .\process.jl:525 [inlined] [2] #run#565(::Bool, ::typeof(run), ::Cmd) at .\process.jl:440 [3] run at .\process.jl:438 [inlined] [4] run_PATH(::Cmd) at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\static_julia.jl:23 [5] (::PackageCompiler.var"#16#18")() at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\static_julia.jl:309 [6] cd(::PackageCompiler.var"#16#18", ::String) at .\file.jl:93 [7] build_shared(::String, ::String, ::Bool, ::String, ::Bool, ::Nothing, ::Nothing, ::Cmd, ::Nothing) at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\static_julia.jl:308 [8] (::PackageCompiler.var"#40#41"{Bool,String})() at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\system_image.jl:46 [9] cd(::PackageCompiler.var"#40#41"{Bool,String}, ::String) at .\file.jl:93 [10] #compile_system_image#39(::Bool, ::typeof(PackageCompiler.compile_system_image), ::String, ::String) at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\system_image.jl:15 [11] #compile_system_image at .\none:0 [inlined] [12] get_backup!(::Bool, ::String) at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\system_image.jl:63 [13] force_native_image! at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\api.jl:123 [inlined] (repeats 2 times) [14] enable_mkl_startup(::String) at C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\src\install.jl:138 [15] top-level scope at C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\deps\build.jl:9 [16] include at .\boot.jl:328 [inlined] [17] include_relative(::Module, ::String) at .\loading.jl:1105 [18] include(::Module, ::String) at .\Base.jl:31 [19] include(::String) at .\client.jl:424 [20] top-level scope at none:5 in expression starting at C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\deps\build.jl:9 caused by [exception 1] IOError: symlink: operation not permitted (EPERM) Stacktrace: [1] uv_error at .\libuv.jl:97 [inlined] [2] symlink(::String, ::String) at .\file.jl:851 [3] probe_symlink_creation(::String) at C:\Users\yahyaaba\.julia\packages\BinaryProvider\kcGxO\src\PlatformEngines.jl:121 [4] #probe_platform_engines!#30(::Bool, ::typeof(BinaryProvider.probe_platform_engines!)) at C:\Users\yahyaaba\.julia\packages\BinaryProvider\kcGxO\src\PlatformEngines.jl:175 [5] probe_platform_engines! at C:\Users\yahyaaba\.julia\packages\BinaryProvider\kcGxO\src\PlatformEngines.jl:169 [inlined] [6] __init__() at C:\Users\yahyaaba\.julia\packages\BinaryProvider\kcGxO\src\BinaryProvider.jl:28 [7] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:692 [8] _require_search_from_serialized(::Base.PkgId, ::String) at .\loading.jl:776 [9] _require(::Base.PkgId) at .\loading.jl:1001 [10] require(::Base.PkgId) at .\loading.jl:922 [11] require(::Module, ::Symbol) at .\loading.jl:917 [12] include at .\boot.jl:328 [inlined] [13] include_relative(::Module, ::String) at .\loading.jl:1105 [14] include(::Module, ::String) at .\Base.jl:31 [15] include(::String) at .\client.jl:424 [16] top-level scope at C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\deps\build.jl:4 [17] include at .\boot.jl:328 [inlined] [18] include_relative(::Module, ::String) at .\loading.jl:1105 [19] include(::Module, ::String) at .\Base.jl:31 [20] include(::String) at .\client.jl:424 [21] top-level scope at none:5 Build shared library "C:\\Users\\yahyaaba\\.julia\\packages\\PackageCompiler\\4yNnV\\sysimg\\backup\\native\\sys.dll": `'C:\Users\yahyaaba\.julia\packages\WinRPM\BfpQI\deps\usr\x86_64-w64-mingw32\sys-root\mingw\bin\gcc.exe' --sysroot 'C:\Users\yahyaaba\.julia\packages\WinRPM\BfpQI\deps\usr\x86_64-w64-mingw32\sys-root' -shared '-DJULIAC_PROGRAM_LIBNAME="C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.dll"' -o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.dll' -Wl,--whole-archive 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.o' -Wl,--no-whole-archive -std=gnu99 '-IC:\Julia-1.3.1-MKL\include\julia' -DJULIA_ENABLE_THREADING=1 '-LC:\Julia-1.3.1-MKL\bin' -Wl,--stack,8388608 -ljulia -lopenlibm -m64 -Wl,--export-all-symbols` ```
aminya commented 4 years ago

A possible solution is posted here: https://sourceforge.net/p/mingw-w64/mailman/message/21864960/

The problem here is the name decoration. MS doesn't extend the public 
symbols in objects by underscores, mingw-w64 does this by default. But you 
can recompile mingw-w64 crt and your app by using option 
-fno-leading-underscores. This should work.
The other solution would be, name the exported method call_me in VS 
_call_me, and reference in gcc just without the leading underscore.

I think the problem is that I have my own mingw compiler on the environment path.

Edit: Removing my own mingw from path didn't solve the issue

Crown421 commented 4 years ago

Currently also dealing with this and I think the problem is more complicated. I have tried pinning PackageCompiler to 0.6.4 and 0.6.3 and it still fails. I have further tried different commits of MKL, specifically

with the pinned version of PackageCompiler, so the problem does not appear to lie their combination.

Further, as you can see in this CI run, MKL builds successfully on Julia 1.0, but then throws a different error when actually using it with VML.

Run julia-actions/julia-buildpkg@latest
julia --color=yes --project -e "using Pkg; if VERSION >= v\"1.1.0-rc1\"; Pkg.build(verbose=true); else Pkg.build(); end"
Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll.
##[error]The process 'julia' failed with exit code 2
##[error]Node run failed with exit code 1

On Julia 1.3 the build itself fails as above.

Crown421 commented 4 years ago

Further, building MKL already failed on nightly Julia at the end of November, as seen here.

The last PackageCompiler update was Dec. 18, and it appears back then it was WinRPM that caused problems.

I tried pinning all packages to the versions in the successful Julia 1.0 run, but no luck so far.

aminya commented 4 years ago

Further, as you can see in this CI run, MKL builds successfully on Julia 1.0, but then throws a different error when actually using it with VML.

This error was fixed on MKL by https://github.com/JuliaComputing/MKL.jl/commit/a316332e3abebbab76a2184952373780b2b35073. See this successful CI for it: https://ci.appveyor.com/project/andreasnoack/mkl-jl/builds/28053628/job/su5qa4uk4mb5nb3k

aminya commented 4 years ago

This combination was the latest successful build on Julia 1 Windows:

Packages ```julia Installed WinRPM ────────── v0.4.2 Installed HTTPClient ────── v0.2.1 Installed Compat ────────── v2.2.0 Installed Libz ──────────── v1.0.0 Installed BinDeps ───────── v0.8.10 Installed BufferedStreams ─ v1.0.0 Installed BinaryProvider ── v0.5.8 Installed PackageCompiler ─ v0.6.4 Installed LibExpat ──────── v0.5.0 Installed URIParser ─────── v0.4.0 Installed LibCURL ───────── v0.5.2 Updating `C:\projects\mkl-jl\Project.toml` [b99e7846] + BinaryProvider v0.5.8 [9b87118b] + PackageCompiler v0.6.4 [8f399da3] + Libdl [37e2e46d] + LinearAlgebra [8dfed614] + Test Updating `C:\projects\mkl-jl\Manifest.toml` [9e28174c] + BinDeps v0.8.10 [b99e7846] + BinaryProvider v0.5.8 [e1450e63] + BufferedStreams v1.0.0 [34da2185] + Compat v2.2.0 [0862f596] + HTTPClient v0.2.1 [b27032c2] + LibCURL v0.5.2 [522f3ed2] + LibExpat v0.5.0 [2ec943e9] + Libz v1.0.0 [9b87118b] + PackageCompiler v0.6.4 [30578b45] + URIParser v0.4.0 [c17dfb99] + WinRPM v0.4.2 [2a0f44e3] + Base64 [ade2ca70] + Dates [8bb1440f] + DelimitedFiles [8ba89e20] + Distributed [b77e0a4c] + InteractiveUtils [76f85450] + LibGit2 [8f399da3] + Libdl [37e2e46d] + LinearAlgebra [56ddb016] + Logging [d6f4376e] + Markdown [a63ad114] + Mmap [44cfe95a] + Pkg [de0858da] + Printf [3fa0cd96] + REPL [9a3f8284] + Random [ea8e919c] + SHA [9e88b42a] + Serialization [1a1011a3] + SharedArrays [6462fe0b] + Sockets [2f01184e] + SparseArrays [10745b16] + Statistics [8dfed614] + Test [cf7118a7] + UUIDs [4ec0a83e] + Unicode ```
IanButterworth commented 4 years ago

Given FFTW.jl now depends on MKL_jll (which includes IntelOpenMP_jll), it seems worth it to try a PR that simplifies the build to just depending on MKL_jll.

The caveat is that jll's require julia 1.3.

Or you could just use the binaries that have been wrapped: https://github.com/JuliaBinaryWrappers/MKL_jll.jl https://github.com/JuliaBinaryWrappers/IntelOpenMP_jll.jl

KristofferC commented 4 years ago

The problem is that you want Julia's own methods (like matrix multiplication) to call into MKL which with the current code structure requires a sysimage rebuild.

aminya commented 4 years ago

The problem is that you want Julia's own methods (like matrix multiplication) to call into MKL which with the current code structure requires a sysimage rebuild.

This is the default MKL.jl repository.

KristofferC commented 4 years ago

Yes... Sorry I didn't understand the point you were making. Could you elaborate?

aminya commented 4 years ago

Yes... Sorry I didn't understand the point you were making. Could you elaborate?

We can't use MKL.jl on Windows:

julia>]add https://github.com/JuliaComputing/MKL.jl

results in the error I wrote in the OP

KristofferC commented 4 years ago

My reply was regarding MKL_jll and that it doesn't change the fact that the sysimage has to be rebuilt.

Crown421 commented 4 years ago

Somewhat off-topic, but who is maintaining MKL_jll/ are packages apart from FFTW.jl using it?

IanButterworth commented 4 years ago

@Crown421 Like most JLL's it's being "maintained" by the #binarybuilder julia slack channel folks. I'm not sure if packages other than FFTW.jl are using it yet.

@KristofferC My bad, I though it was a MKL build problem rather than a PackageCompiler issue. Should've read the issue in more detail. Sorry for the noise

aminya commented 4 years ago

So should we switch to PackageCompilerX or something or we can solve this in an easier way?

@KristofferC

KristofferC commented 4 years ago

I don't think the error Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll would be helped by doing that change.

jlperla commented 4 years ago

Is moving to require Julia 1.3 so you rely on the artifacts helpful? I think it is a reasonable requirement if it would simplify the setup issues of binary locations?

aminya commented 4 years ago

I don't think the error Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll would be helped by doing that change.

This is not the error that i get. Please see the log I attached in OP.