Add LAPACK functionality

[dotChris90]

In order to reduce the complexity and the code we have to write we should think about using a native lib LAPACK. If we can use native lib we would have a lot easier for QR, inverse, SVD, EVD, etc.

This issue is for keeping things and ideas we had.

@fdncred @Oceania2018 @vunb if anybody of you have an idea or suggest - write it down ;)

[dotChris90]

I tried first of all http://www.netlib.org/lapack/ which is even compiled for windows. The problem I have is that I can not use PInvoke for it. C# can not find the dll even it lay beside. Usual this means others dll are missing. I think gcc based libraries .... or it even expect that gcc libs are on the system.

[dotChris90]

I found out that portable pythons have their own lapack dll which can be detected by python. https://portablepython.com/wiki/Download/ I will maybe look if there is a lapack lite dll for x64. ;)

[fdncred]

Look at this first https://github.com/Proxem/BlasNet

[dotChris90]

ok that is an option. is this MKL open source? :D

[dotChris90]

anybody has experience with https://sourceforge.net/projects/arma/ ? the download brings a dll which is at least visible for the PInvoke - will check now if we can invoke.

[fdncred]

Here's the MKL link. https://software.intel.com/en-us/mkl

[dotChris90]

ah guys. the Armadillo Lib from source forge link brings a compiled version of lapack. and pinvoke can find it. until now I got some cast errors but that's OK. better as before.

The Intel library we can try out too but I am not sure how popular they are in open source world. the lapack from armadillo is the default open source version which is used in most Libraries. but we could deliver 2 versions and user can choose. if this Intel library can be used without installation. :)

[dotChris90]

@fdncred @Oceania2018

success! YEAH! I share my experience so far.

The Armadillo Lib from source forge brings an compiled, native DLL of LAPACK. According to their Readme it is the compiled version of http://www.netlib.org/lapack/. So the one and only original LAPACK. I tested just x64 so far and on Windows 10 - no linux tests or x86.

Files In Explorer

The DLLViewer showed me the correct LAPACK functions inside so I simple checked their documentation and selected dgesv_ function to solve linear systems. And here it is!

DLLWrapper

If you 2 do not mind - I had created a new project beside NumSharp.Core - called NumSharp.LAPACK. NumSharp.Core will use NumSharp.LAPACK. Unit Tests for NumSharp.LAPACK are 2nd separate MSTEST Prokect. I will do a pull request and merge to our main Numsharp Repo. ;)

[Oceania2018]

Congrats.

[fdncred]

I did some research this morning and I'm still in the camp that MKL should be used, unless you plan on running on AMD processors. In that case, we should use OpenBLAS. MKL specifically tuned and optimized by hand for Intel processors and is much faster than just about anything else, and it's free.

Some benchmarks http://markus-beuckelmann.de/blog/boosting-numpy-blas.html https://software.intel.com/en-us/mkl/features/benchmarks https://software.intel.com/en-us/mkl/features/linear-algebra https://github.com/tmolteno/necpp/issues/18

Here's the documentation for the LAPACK routines in MKL. https://software.intel.com/en-us/mkl-developer-reference-c-lapack-routines

Since Proxem has already implemented MKL I'm not sure why we wouldn't use it, unless it doesn't do what we need it to do.

However, anything will be better than our own code. Any of these libraries have spent a ton of time trying to get it right.

[fdncred]

Armadillo looks like a nice wrapper to all of these libraries too. Code is on GitLab too. However, we'd have to P/Invoke it too.

This is interesting http://www.dotnumerics.com/NumericalLibraries/LinearAlgebra/CSLapack/Default.aspx http://www.dotnumerics.com/NumericalLibraries/LinearAlgebra/CSBlas/Default.aspx

Wikipedia's list of packages https://en.wikipedia.org/wiki/Comparison_of_linear_algebra_libraries

[dotChris90]

@fdncred thanks for the explaining. Sure MKL is the best performance option but on intel processors which is a minus point. The important thing in todays time is not - its free or not - the most important is - is it open Source? ^^

On Top of this numpy do not use the intel lib as standard. They use the default implementation. About Anaconda I am not sure ....

My dream is that the users can choose free. As far as I read all this libraries implement LAPACK and this mean - all their APIS must follow the same structure. So later we can use both.

So in future we can use the same PInvoke but with different native DLLs or SO - this would be really free choose for NumSharp.

First I simple want to run NumSharp. No matter with optimal or non optimal performance. ;) But yes MKL shall be included also - by the way MKL is also simple a dll or need a installation?

[fdncred]

@dotChris90, MKL is just a dll or two. You have to install it to get those dlls i.e. you can't just download the dlls. But once you have the dlls you could use them without the installation environment.

If your goal is to let the end user decide which they want, then it seems like Armadillo would be best since it supports most of the various opensource LAPACK, BLAS, MKL, OpenBlas, etc.

It doesn't make any difference to me. I'm just providing information about the tools available.

[dotChris90]

all ok :)

i think this Intel and AMD libraries is a must for performance after our Apis was implemented. maybe i should install them and see how we can use them in parallel. :)

and every information is a good information ;)

[dotChris90]

close because lapack is now part of it - just need to be better --> performance etc. is traced in #116.