mathnet / mathnet-numerics

Math.NET Numerics
http://numerics.mathdotnet.com
MIT License
3.44k stars 893 forks source link

MathNet.Numerics 4.15.0 high memory consumption Frobenius on Windows Docker linux container #1076

Open jazzikowy opened 1 month ago

jazzikowy commented 1 month ago

Hi,

I've observed that my builds are breaking with out of memory exception when I run them on Docker Linux containers. The code snippet below for 1st TestCase is throwing the out of memory exception consuming more than 41GB of memory during run (my container is configured to have 12GB memory limit but this operation shouldn't exceed 8MB) and runs for more than 2min. The second one works in 300ms and runs without any issue. Unfortunately I cannot update my package yet to 5.0.0 did someone experience similar issues? Was it fixed in newer version?

[TestCase(42737, 17)]
[TestCase(17, 42737)]
public void FrobeniusNormOfRandomMatrix(int rows, int columns)
{
    var testMatrix = (DenseMatrix)DenseMatrix.Build.Random(rows, columns, 42);
    var matrixCopy = testMatrix.Clone();

    (testMatrix - matrixCopy).FrobeniusNorm().ShouldBe(0.0, 1e-15);
}
AngxiaoLu123 commented 2 weeks ago

Hi there @jazzikowy , I'm not able to install MKL in my Linux docker container, could you please teach me how did you install it? Thanks a lot. I'm using .NET 8 and nuget package MathNet.Numerics.MKL.Linux 3.0.0-beta3, but when I call Control.UseNativeMKL(), it throws this:

System.DllNotFoundException: Unable to load shared library 'libMathNetNumericsMKL' or one of its dependencies. In order to help diagnose loading problems, consider using a tool like strace. If you're using glibc, consider setting the LD_DEBUG environment variable: 
libiomp5.so: cannot open shared object file: No such file or directory
/usr/share/dotnet/shared/Microsoft.NETCore.App/8.0.6/libMathNetNumericsMKL.so: cannot open shared object file: No such file or directory
/app/libMathNetNumericsMKL.so: cannot open shared object file: No such file or directory
/app/runtimes/linux-x64/native/liblibMathNetNumericsMKL.so: cannot open shared object file: No such file or directory
/usr/share/dotnet/shared/Microsoft.NETCore.App/8.0.6/liblibMathNetNumericsMKL.so: cannot open shared object file: No such file or directory
/app/liblibMathNetNumericsMKL.so: cannot open shared object file: No such file or directory
/app/runtimes/linux-x64/native/libMathNetNumericsMKL: cannot open shared object file: No such file or directory
/usr/share/dotnet/shared/Microsoft.NETCore.App/8.0.6/libMathNetNumericsMKL: cannot open shared object file: No such file or directory
/app/libMathNetNumericsMKL: cannot open shared object file: No such file or directory
/app/runtimes/linux-x64/native/liblibMathNetNumericsMKL: cannot open shared object file: No such file or directory
/usr/share/dotnet/shared/Microsoft.NETCore.App/8.0.6/liblibMathNetNumericsMKL: cannot open shared object file: No such file or directory
/app/liblibMathNetNumericsMKL: cannot open shared object file: No such file or directory

   at MathNet.Numerics.Providers.MKL.SafeNativeMethods.query_capability(Int32 capability)
   at MathNet.Numerics.Providers.MKL.MklProvider.Load(String hintPath, MklConsistency consistency, MklPrecision precision, MklAccuracy accuracy)
   --- End of inner exception stack trace ---
   at MathNet.Numerics.Providers.MKL.MklProvider.Load(String hintPath, MklConsistency consistency, MklPrecision precision, MklAccuracy accuracy)
   at MathNet.Numerics.Providers.MKL.LinearAlgebra.MklLinearAlgebraProvider.InitializeVerify()
   at MathNet.Numerics.Providers.LinearAlgebra.LinearAlgebraControl.set_Provider(ILinearAlgebraProvider value)
   at MathNet.Numerics.Providers.LinearAlgebra.LinearAlgebraControl.UseNativeMKL()