search

dotnet / machinelearning

ML.NET is an open source and cross-platform machine learning framework for .NET.
https://dot.net/ml
MIT License
9.04k stars 1.89k forks source link

Unable to load shared library 'MklImports' or one of its dependencies. #3903

Closed mphollas closed 4 years ago

mphollas commented 5 years ago

System information

Issue

Source code / logs

Unable to load shared library 'MklImports' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable: dlopen(libMklImports, 1): image not found

at Microsoft.ML.Transforms.TimeSeries.EigenUtils.Dsytrd(Layout matrixLayout, Uplo uplo, Int32 n, Double[] a, Int32 lda, Double[] d, Double[] e, Double[] tau)\n at Microsoft.ML.Transforms.TimeSeries.EigenUtils.MklSymmetricEigenDecomposition(Single[] input, Int32 size, Single[]& eigenValues, Single[]& eigenVectors)\n at Microsoft.ML.Transforms.TimeSeries.TrajectoryMatrix.ComputeSvd(Single[]& singularValues, Single[]& leftSingularvectors)\n at Microsoft.ML.Transforms.TimeSeries.AdaptiveSingularSpectrumSequenceModeler.AdaptiveSingularSpectrumSequenceModelerInternal.TrainCore(Single[] dataArray, Int32 originalSeriesLength)\n at Microsoft.ML.Transforms.TimeSeries.AdaptiveSingularSpectrumSequenceModeler.AdaptiveSingularSpectrumSequenceModelerInternal.Train(RoleMappedData data)\n

ilias commented 5 years ago

I got a similar error running an ASP.NET core 2.2 app on a linux64 docker container

fail: Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware[1] An unhandled exception has occurred while executing the request. System.DllNotFoundException: Unable to load shared library 'MklImports' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libMklImports: cannot open shared object file: No such file or directory at Microsoft.ML.Transforms.TimeSeries.EigenUtils.Dsytrd(Layout matrixLayout, Uplo uplo, Int32 n, Double[] a, Int32 lda, Double[] d, Double[] e, Double[] tau) at Microsoft.ML.Transforms.TimeSeries.EigenUtils.MklSymmetricEigenDecomposition(Single[] input, Int32 size, Single[]& eigenValues, Single[]& eigenVectors) at Microsoft.ML.Transforms.TimeSeries.TrajectoryMatrix.ComputeSvd(Single[]& singularValues, Single[]& leftSingularvectors) ….

flo8 commented 5 years ago

Hi!

I run into the same exact situation with a .Net Core 2.2 app (console) here using Ubuntu 16.04 x64: Description: Ubuntu 16.04.6 LTS Release: 16.04 Codename: xenial

dotnet --version 2.2.300

I get exactly the same exception as described above.

eduardobr commented 5 years ago

Same issue in here. mlnet 0.14.27903.10

.NET Core SDK (reflecting any global.json): Version: 2.2.401 Commit: 729b316c13

Runtime Environment: OS Name: Mac OS X OS Version: 10.14 OS Platform: Darwin RID: osx.10.14-x64 Base Path: /usr/local/share/dotnet/sdk/2.2.401/

Host (useful for support): Version: 2.2.6 Commit: 7dac9b1b51

.NET Core SDKs installed: 2.2.301 [/usr/local/share/dotnet/sdk] 2.2.401 [/usr/local/share/dotnet/sdk]

.NET Core runtimes installed: Microsoft.AspNetCore.All 2.2.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All] Microsoft.AspNetCore.All 2.2.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All] Microsoft.AspNetCore.App 2.2.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 2.2.6 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

Benzidrine commented 5 years ago

Same issue here

OS Name: Mac OS X OS Version: 10.14.1 Platform: Darwin

dotnet --version 2.2.300

wschin commented 5 years ago

@codemzs, any comment?

archeg commented 5 years ago

I have the same issue, but on Windows and 4.6 .Net framework.

OS: Windows 10

.NET Framework 4.6.1

Trying to use Anomaly Detection, only Iid works, SSA and SRCNN do not. On the same machine, if running on .Net Core 2.2 everything works well

flo8 commented 5 years ago

Hi @wschin @codemzs !

Is there anything we can do to help you guys move this forward?

Many thanks,

dhunt84971 commented 5 years ago

Having same issue on Ubuntu 19.04 with AnomalyDetection_PowerMeterReadings.

dotnet --version 2.2.401

mlnet 0.15.28007.4

flo8 commented 5 years ago

Hi @codemzs @wschin !

How can we help you troubleshoot this?

Cheers

lessenko commented 5 years ago

Hi! I have the same issue.

Runtime Environment: RID: ubuntu.18.04-x64 Base Path: /usr/share/dotnet/sdk/2.2.401/

I tried the sample "DetectChangePointBySsaStream.cs" for ML 1.2.0, 1.3.1 and 1.4.0-preview

Thanks

rpln commented 5 years ago

Had the precise error described here when using TimeSeries in ML.net and what fixed it for me was installing the Intel MKL: https://software.intel.com/en-us/mkl

In my case specifically, I was working with dotnet via the .NET Core SDK Docker images: https://hub.docker.com/_/microsoft-dotnet-core-sdk/

Within docker (debian base), the fix for me was:

RUN \
  sed -i 's/ main/ main contrib non-free/' /etc/apt/sources.list && \
  apt-get update && \
  DEBIAN_FRONTEND=noninteractive apt-get install -y intel-mkl

I presume similar installs of Intel MKL on non-debian linux or macos are also likely to fix the issue

flo8 commented 5 years ago

Hi @rpln and thanks for sharing this info!

I also tried it here on our systems but still have the problem. Is there a way to make sure that MKL is correctly installed?

I also didn't find any mention of how to install this dependency on ML.Net website. Trying to Google it didn't give much results as well. @codemzs any idea on this?

Many thanks,

flo8 commented 5 years ago

Hi all!

Thanks a lot @rpln, after checking this link on SO it seemed that MKL was not correctly installed in our Docker container even if we followed the instructions above. After following this and this, it now seems to work correctly.

It's however a bit sad to see that this MKL thing is currently nowhere to be found on ML.Net documentation and that this ticket has been opened for almost 4 months :(

Thanks a lot to @rpln for your contribution!

Flo

lessenko commented 5 years ago

Hi all!

Thanks a lot @rpln and @flow8 my sample works. My decision is to put library libiomp5.so from intel-mkl 2019.0.117 into local nuget directory for microsoft.ml.mkl.redist. Docker is not used.

Regards.

AceHack commented 5 years ago

I can't get this working on Mac.

I tried installing MKL from https://software.seek.intel.com/performance-libraries

But it's still failing.

AceHack commented 5 years ago

I've tried following the instructions here too, no luck https://github.com/firemodels/fds/wiki/Linking-to-Intel-Math-Kernel-Libraries-in-Linux-and-OSX https://github.com/firemodels/fds/wiki/Compiling-FDS-with-GNU-Fortran-in-Mac-OSX

lessenko commented 5 years ago

Hi AceHack! Just in case, libiomp5 has needed definitions for libMklImports (Nuget). It's for Ubuntu. For Mac I suppose it's analog libMklImports.dylib and libiomp5.dylib https://software.intel.com/en-us/articles/dyld-library-not-loadedlibiomp5dylib

Nuget local dir for Mac ---.nuget/packages/microsoft.ml.mkl.redist/1.3.1/runtimes/osx-x64/native/

AceHack commented 5 years ago

I am running the mlnet global tool, microsoft.ml.mkl.redist is not present in my nuget cache.

lessenko commented 5 years ago

Just in case, I use Mono, MonoDevelop and Nuget (in MonoDevelop).

lessenko commented 5 years ago

Via NuGet I add only Microsoft.ML.TimeSeries package. Microsoft.ML.Mkl.Redist is added itself.

danjagnow commented 5 years ago

I'm seeing the same thing using Microsoft.ML.TimeSeries 1.3.1 on Windows 10 Enterprise, with a library targeting net48;netstandard2.0 and a unit test project targeting net48.

System.TypeInitializationException : The type initializer for 'Microsoft.ML.Transforms.TimeSeries.FftUtils' threw an exception.
  ----> System.DllNotFoundException : Unable to load DLL 'MklImports': The specified module could not be found. (Exception from HRESULT: 0x8007007E)
   at Microsoft.ML.Transforms.TimeSeries.FftUtils.ComputeForwardFft(Single[] inputRe, Single[] inputIm, Single[] outputRe, Single[] outputIm, Int32 length)
   at Microsoft.ML.Transforms.TimeSeries.SrCnnAnomalyDetectionBase.SrCnnAnomalyDetectionBaseCore.State.SpectralResidual(Single input, FixedSizeQueue`1 data, VBufferEditor`1& result)
   at Microsoft.ML.Transforms.TimeSeries.SrCnnTransformBase`2.SrCnnStateBase.TransformCore(TInput& input, FixedSizeQueue`1 windowedBuffer, Int64 iteration, VBuffer`1& dst)
   at Microsoft.ML.Transforms.TimeSeries.SequentialTransformerBase`3.StateBase.Process(TInput& input, TOutput& output)
   at Microsoft.ML.Transforms.TimeSeries.SrCnnTransformBase`2.Mapper.<>c__DisplayClass16_0.<MakeGetter>b__0(VBuffer`1& dst)
   at Microsoft.ML.Data.TypedCursorable`1.TypedRowBase.<>c__DisplayClass8_0`1.<CreateDirectVBufferSetter>b__0(TRow row)
   at Microsoft.ML.Data.TypedCursorable`1.TypedRowBase.FillValues(TRow row)
   at Microsoft.ML.Transforms.TimeSeries.TimeSeriesPredictionEngine`2.Predict(TSrc example, TDst& prediction, Nullable`1 horizon, Nullable`1 confidenceLevel)
   at Microsoft.ML.Transforms.TimeSeries.TimeSeriesPredictionEngine`2.Predict(TSrc example, Nullable`1 horizon, Nullable`1 confidenceLevel)
lessenko commented 5 years ago

I've tried the sample in VS 2017 community on Windows 7 and 10 (dotnet 2.1) at last weekend. It seems it worked without problem.

AceHack commented 5 years ago

Still having issue on Mac.

todarko commented 4 years ago

ML.net 1.4 / Docker container with Ubuntu 16.04 .net core 3.0

Works find debugging locally on Windows, but deploying it to Ubuntu container throws the error intermittently. Half the time it works, half the time it throws the error. It's not random and throws on certain data sets.

System.DllNotFoundException: Unable to load shared library 'MklImports' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libMklImports: cannot open shared object file: No such file or directory web_1 | at Microsoft.ML.Transforms.TimeSeries.EigenUtils.Dsytrd(Layout matrixLayout, Uplo uplo, Int32 n, Double[] a, Int32 lda, Double[] d, Double[] e, Double[] tau) web_1 | at Microsoft.ML.Transforms.TimeSeries.EigenUtils.MklSymmetricEigenDecomposition(Single[] input, Int32 size, Single[]& eigenValues, Single[]& eigenVectors) web_1 | at Microsoft.ML.Transforms.TimeSeries.TrajectoryMatrix.ComputeSvd(Single[]& singularValues, Single[]& leftSingularvectors) web_1 | at Microsoft.ML.Transforms.TimeSeries.AdaptiveSingularSpectrumSequenceModelerInternal.TrainCore(Single[] dataArray, Int32 originalSeriesLength) web_1 | at Microsoft.ML.Transforms.TimeSeries.AdaptiveSingularSpectrumSequenceModelerInternal.Train(RoleMappedData data) web_1 | at Microsoft.ML.Transforms.TimeSeries.SsaForecastingTransformer..ctor(IHostEnvironment env, Options options, IDataView input) web_1 | at Microsoft.ML.Transforms.TimeSeries.SsaForecastingEstimator.Fit(IDataView input) web_1 | at Veritas_Web.Utilities.MLTSVFile.ForcastWithTime(Int32 size, IDataView dataView) in /src/Utilities/MLTSVFile.cs:line 211

hectorcubillos commented 4 years ago

Same issue on .NET Core 3.1

Version: 3.1.101 Runtime Environment: OS Name: Mac OS X OS Version: 10.15 OS Platform: Darwin RID: osx.10.15-x64 Base Path: /usr/local/share/dotnet/sdk/3.1.101/

Exception has occurred: CLR/System.DllNotFoundException An unhandled exception of type 'System.DllNotFoundException' occurred in Microsoft.ML.TimeSeries.dll: 'Unable to load shared library 'MklImports' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable: dlopen(libMklImports, 1): image not found'

lclamotte commented 4 years ago

@hectorcubillos and everyone else, I was able to get this fixed

Make sure you have the Microsoft.ML.Mkl.Redist nuget package for the project you are attempting to run. For example, right click the C# project in Solution Explorer > Manage Nuget Packages then search for "mkl" and install Microsoft.ML.Mkl.Redist then rebuild your project.

najeeb-kazmi commented 4 years ago

@mphollas @ilias @flo8 @eduardobr @Benzidrine @archeg @dhunt84971 @lessenko @todarko @hectorcubillos

First of all, apologies that it took us this long to get to this. I have begun taken a look at this and here's what I have found so far:

  1. On macOS 10.14.16, I could not reproduce this issue at all. I used .NET Core 2.2.300, which is the most frequently cited version here. I did the following with both Visual Studio for Mac and the dotnet command line: I copied the code from the forecasting sample linked by OP into a new .NET Core console project, then built and ran the project with ML.NET 1.1, 1.2, 1.3.1, 1.4, 1.5.0-preview2. In all 2x5=10 attempts, I could not reproduce the error. If one of you could provide me with the smallest .NET Core project that can reproduce this, along with details about your environment, I could investigate further.

  2. I was able to reproduce this on Ubuntu 18.04 with both .NET Core 2.2.300 and 3.1.200. I created a new project with the dotnet command line, and followed the same steps as for macOS above. The issue was reproduced for all five versions of ML.NET with dotnet run. The native dlls were not present in bin/Debug/netcoreapp2.2, and with dotnet publish, not copied into bin/Debug/netcoreapp2.2/publish folder. This persisted even after adding NativeAssemblyReference to the project.

    @eerhardt does this situation look familiar to you? Here are the contents of my .csproj file:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp2.2</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.ML" Version="1.2" />
    <PackageReference Include="Microsoft.ML.Mkl.Redist" Version="1.2" />
    <PackageReference Include="Microsoft.ML.TimeSeries" Version="1.2" />
    <NativeAssemblyReference Include="CpuMathNative" />
    <NativeAssemblyReference Include="FastTreeNative" />
    <NativeAssemblyReference Include="MatrixFactorizationNative" />
    <NativeAssemblyReference Include="LdaNative" />
    <NativeAssemblyReference Include="SymSgdNative" />
    <NativeAssemblyReference Include="MklProxyNative" />
  </ItemGroup>

</Project>

cc: @harishsk

harishsk commented 4 years ago

@najeeb-kazmi Is your project part of ML.NET sources or a standalone project? Can you turn on verbose logging in your msbuild and see if the logging has any mention of trying to copy these files?

If your project is a standalone project I suspect they aren't being copied over because they are missing the CopyNativeAssemblies target that is present in Directory.Build.Targets in ML.NET.

eerhardt commented 4 years ago

NativeAssemblyReference is internal to the dotnet/machinelearning repo. This isn't something that users would ever use. We only do it in our repo to get native assemblies placed in our bin directory.

For anyone who is running into this issue on a mac, there are 2 things I can recommend.

  1. To try to debug why this is happening, the exception message says In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable: dlopen(libMklImports, 1). Consider doing this and sharing the output. It would help figure out what is going wrong.
  2. An option to try to fix it is to install libomp on the machine. This is what we do on our CI servers:
brew update && brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/f5b1ac99a7fba27c19cee0bc4f036775c889b359/Formula/libomp.rb && brew link libomp --force
harishsk commented 4 years ago

It appears that the tests and samples in the ML.NET repo are relying on NativeAssemblyReference to make sure that the native assemblies are copied over. Since this facility is not available outside the repo, what is our recommendation to developers taking a dependency on the nugets directly for making sure that the native assemblies are copied over?

eerhardt commented 4 years ago

what is our recommendation to developers taking a dependency on the nugets directly for making sure that the native assemblies are copied over?

They reference our nuget packages, and the .NET tooling (NuGet, MSBuild) copies the native assemblies correctly.

najeeb-kazmi commented 4 years ago

On Ubuntu 16.04 and 18.04, in my standalone project with Microsoft.ML.Timeseries nuget, libMklmports.so is present in /bin/Debug/netcoreapp3.1/runtimes/linux-x64/native, but it has a dependency on libiomp5.so, which is not found. Following the instructions here, I installed MKL, where in the "How do I install a particular version?", I installed the latest version for my 64 bit distro:

$ sudo apt-get install intel-mkl-64bit-2020.0-088

This installs libiomp in /opt/intel. Then, the location where libiomp5 gets installed:

$ find /opt -name "libiomp5.so"
/opt/intel/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64_lin/libiomp5.so

needs to be added to the load library path:

$ sudo ldconfig /opt/intel/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64_lin/

After this, I can run my console app without an error.

najeeb-kazmi commented 4 years ago

On the other hand, on macOS 10.14.6, the libraries are present:

dyld: loaded: /Users/admin/.nuget/packages/microsoft.ml.mkl.redist/1.5.0-preview2/runtimes/osx-x64/native/libMklImports.dylib
dyld: loaded: /usr/local/lib/libiomp5.dylib

On Windows, the dll gets copied to and loaded from \bin\Debug\netcoreapp3.0\runtimes\win-x64\native\libiomp5md.dll.

najeeb-kazmi commented 4 years ago

So the root cause is that libMklImports depends on OpenMP library libiomp5. Because of redistribution licensing reasons, libiomp5.dll is packaged in the ML.NET nugets for Windows only. libiomp5.dylib and libiomp5.so are not packaged for MacOS and Linux, respectively. If these libraries are not already present on the system, this error will be thrown.

Note: libMklImports itself is packaged for Windows, MacOS, and Linux.

For now, the solution is to install the library directly. We will make a decision on whether to get redistribution licensing for libiomp5, and decide where to document this in the meantime.

MacOS

Install libiomp5 version 7 specifically.

brew update && brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/f5b1ac99a7fba27c19cee0bc4f036775c889b359/Formula/libomp.rb && brew link libomp --force

This should download the library to /usr/local/lib/libiomp5.dylib, from where it should be loaded without an issue.

Linux

Install MKL, which will install libiomp.so, following the instructions here, specifically these steps:

Install the GPG key for the repository:

sudo bash
# <type your user password when prompted.  this will put you in a root shell>
# cd to /tmp where this shell has write permission
cd /tmp
# now get the key:
wget https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB
# now install that key
apt-key add GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB
# now remove the public key file exit the root shell
rm GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB
exit

Add the APT Repository for MKL:

sudo sh -c 'echo deb https://apt.repos.intel.com/mkl all main > /etc/apt/sources.list.d/intel-mkl.list'

Update packages:

sudo apt-get update

Install MKL:

sudo apt-get install <COMPONENT>-<VERSION>.<UPDATE>-<BUILD_NUMBER>

For example:

sudo apt-get install intel-mkl-64bit-2020.0-088

This installs libiomp.so in /opt/intel. Find the exact location:

find /opt -name "libiomp5.so"
/opt/intel/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64_lin/libiomp5.so

Add this location to the load library path:

sudo ldconfig /opt/intel/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64_lin/
najeeb-kazmi commented 4 years ago

These additional steps have been documented here: https://docs.microsoft.com/en-us/dotnet/machine-learning/how-to-guides/install-extra-dependencies

MarinaBodrasheva commented 4 years ago

Hi guys! Is there any way to fix it for Windows? Running the app in Docker windows image

I have installed all required nuget packages (Microsoft.ML.TimeSeries, Microsoft.ML.AutoML) but still the same error =(... any suggestion @najeeb-kazmi ?

chris-steema commented 4 years ago

I'm getting this error too using DetectSpikeBySsa in VS2019 on Windows 10 (C#) - I have the Microsoft.ML.Mkl.Redist NuGet installed.

Anomaly_TeeChart_2020-04-28_14-38-13

praveenraghuvanshi1512 commented 4 years ago

I am getting 'Unable to load DLL 'MklImports' or one of its dependencies' while calling Fit method of ForecastBySsa() in a time series prediction.

The issue surfaced in Jupyter notebook only, the same code runs fine in Visual Studio 2019.

OS: Windows 10 x64 .Net core: 3.1.300 Microsoft.ML : 1.5.0 Microsoft.ML.TimeSeries : 1.5.0 Visual Studio 2019 16.6.0

Logs SubmitCode: var model = pipeline.Fit(data); CodeSubmissionReceived: var model = pipeline.Fit(data); CompleteCodeSubmissionReceived: var model = pipeline.Fit(data); System.DllNotFoundException: Unable to load DLL 'MklImports' or one of its dependencies: The specified module could not be found. (0x8007007E) at Microsoft.ML.Transforms.TimeSeries.EigenUtils.Dsytrd(Layout matrixLayout, Uplo uplo, Int32 n, Double[] a, Int32 lda, Double[] d, Double[] e, Double[] tau) at Microsoft.ML.Transforms.TimeSeries.EigenUtils.MklSymmetricEigenDecomposition(Single[] input, Int32 size, Single[]& eigenValues, Single[]& eigenVectors) at Microsoft.ML.Transforms.TimeSeries.TrajectoryMatrix.ComputeSvd(Single[]& singularValues, Single[]& leftSingularvectors) at Microsoft.ML.Transforms.TimeSeries.AdaptiveSingularSpectrumSequenceModelerInternal.TrainCore(Single[] dataArray, Int32 originalSeriesLength) at Microsoft.ML.Transforms.TimeSeries.AdaptiveSingularSpectrumSequenceModelerInternal.Train(RoleMappedData data) at Microsoft.ML.Transforms.TimeSeries.SsaForecastingTransformer..ctor(IHostEnvironment env, Options options, IDataView input) at Microsoft.ML.Transforms.TimeSeries.SsaForecastingEstimator.Fit(IDataView input) at Submission#30.<>d__0.MoveNext()

danielleiszen commented 4 years ago

These additional steps have been documented here: https://docs.microsoft.com/en-us/dotnet/machine-learning/how-to-guides/install-extra-dependencies

The documentation is clear, however it does not work for me. I get the following exception when calling DetectSpikeBySsa

System.TypeInitializationException: 'The type initializer for 'Microsoft.ML.Transforms.TimeSeries.FftUtils' threw an exception.' DllNotFoundException: Unable to load shared library 'MklImports' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libMklImports: cannot open shared object file: No such file or directory

when I run find /opt -name "libiomp5.so" , I get: /opt/intel/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64_lin/libiomp5.so - so the installation is successfull.

And one little comment about the exception: the LD_DEBUG env variable does not have a lib option, it's libs I use the mcr.microsoft.com/dotnet/core/aspnet:3.1-buster-slim container.

Please, let me know what I can do in order to resolve this problem. Thank you for your kind help.

frank-dong-ms-zz commented 4 years ago

@danielleiszen did you add libiomp path to the load library path like below? sudo ldconfig /opt/intel/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64_lin

you can verify this by check the "LD_LIBRARY_PATH" variable

tsustun commented 4 years ago

Hi. I have finally resolved this issue on:

Windows 10 Pro Visual Studio 2019 Microsoft.ML 1.5.2 Microsoft.ML.Mkl.Redist 1.5.2 Microsoft.ML.TimeSeries 1.5.2

For me, the solution was to select "x86", rather than "AnyCPU", as Platform Target. The DLL's seem to be copied to \bin directory successfully when either "x86" or "x64" is selected. Hope this helps someone...

Abouzeid commented 4 years ago

I also faced this issue on Windows 10 machine. The app worked fine in Visual studio 2019 but failed with DllNotFoundException after I deployed it to another machine.

I can locatet the dll in the bin folder but the app can not. It turned out that it is a depecdency issue what was missing was other DLLs not the MklImports.

Here is the depecdency check output:

Dump of file MklImports.dll

File Type: DLL

Image has the following dependencies:

KERNEL32.dll
VCRUNTIME140.dll
api-ms-win-crt-heap-l1-1-0.dll
api-ms-win-crt-string-l1-1-0.dll
api-ms-win-crt-convert-l1-1-0.dll
api-ms-win-crt-stdio-l1-1-0.dll
api-ms-win-crt-runtime-l1-1-0.dll
libiomp5md.dll
api-ms-win-crt-math-l1-1-0.dll
api-ms-win-crt-time-l1-1-0.dll

The solution besides what other said:

-Install Microsoft Visual C++ Redistributable from here(https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads)

RobotOptimist commented 3 years ago

I wish there were a way to remove this dependency as this makes running ML.NET from a notebook on Linux quite painful. This is particularly true when trying to use it on mybinder.org for sharing notebooks.

I've created a base image for this task jmacivor/dotnet-binder:0.1.1 which will load in that dependency and allows you to run ML.NET from a Linux Dockerfile. This still uses .NET 3.1 however, as I haven't yet upgraded to .NET 5.

Still, my preference would be for this to work natively and easily from Linux.

eerhardt commented 3 years ago

I wish there were a way to remove this dependency

@RobotOptimist - the MKL dependency isn't necessary for the base ML.NET libraries. Are you using the Microsoft.ML.AutoML library? That brings in a dependency on MKL by default. AutoML uses MKL for 2 trainers:

If you remove these trainers in your ExperimentSettings, you shouldn't need MKL on the machine. For example:

            BinaryExperimentSettings settings = new BinaryExperimentSettings();
            settings.MaxExperimentTimeInSeconds = 60;
            settings.Trainers.Remove(BinaryClassificationTrainer.SymbolicSgdLogisticRegression);

            ExperimentResult<BinaryClassificationMetrics> experimentResult = mlContext.Auto()
                .CreateBinaryClassificationExperiment(settings)
                .Execute(trainDataView);
RobotOptimist commented 3 years ago

It is Ols that has the problem. It's a shame I have to exclude it unless I install the dependency since it has worked really well for some of my regression models.

Also, I didn't know that AutoML (are you referring to the commandline tool here?) was configurable! I'll be searching for more information on that, it sounds interesting.

eerhardt commented 3 years ago

@RobotOptimist - I opened https://github.com/dotnet/machinelearning/issues/5576 as a suggestion to skip/ignore the trainers that don't work. Thanks for the feedback!

pjsteele commented 3 years ago

I got it to work on OSX, Visual Studio by adding the following nuget package


<PackageReference Include="MKL.NET.osx-x64" Version="2020.4.301" />
leobasilio commented 3 years ago

I got it to work on OSX, Visual Studio by adding the following nuget package

<PackageReference Include="MKL.NET.osx-x64" Version="2020.4.301" />

I can confirm it also works with .NET 5 on Linux. Adding MKL.NET.linux-x64 as a dependency solves the issue.

Gregordy commented 3 years ago

Yes, adding MKL.NET.osx-x64 also works for me on macOS Catalina 10.15.7