Pull linalg lib into a separate repository

[lambday]

I think we should separate our internal linalg lib into a different repository. We can then have

• the master branch (stable API) pulled into shogun main as a submodule
• a develop branch where we can do crazy experimental stuffs

IMO it separates the concerns of having to worry about maths related things and actual ML algorithm related things at the same time. The child repo can have

• its own unit-testing suite
• benchmarks and plots (most important), which would help the shogun devs to choose during implementation/improvement of shogun algorithms

Let me know your thoughts on that @sonney2k @karlnapf @lisitsyn @vigsterkr @besser82 @iglesias @yorkerlin @tklein23

[besser82]

So from my POV this is a good idea; I'm always open for splitting stuff and modularizing. The only thing I would do differently: Instead of having a sub-module in for this in Shogun, I'd transform the whole thing into a separate shared lib, so we can release it decoupled from Shogun and give improvements straight to the users.

Looking in the future, we're planning to pluginize Shogun… So we could do with linalg, too and provide the corresponding different backends (Eigen3, ViennaCL, …) as plugins…

If we split out the linalg-thing and make it more versatile, I'd propose to give it a different name to avoid possible clashes with other projects… My proposal for that would be sth. like "mathalicious" or some other crazy name.

During the split-out process we could also apply the new BSD-3 License to it, just to have that part done already.

[lambday]

@besser82 I really like this proposal. There are many static maths things that we can pull out from Shogun and put it there in a separate mathlib thing as well. Those things might be useful for other people as well. Plus it will make shogun lightweight - no need to compile those maths thing every time you compile shogun since they don't change.

I don't know what do you have in mind to make it a plugin though. Maybe shogun lib should be shipped with this mathlib as well.

+1 for using BSD-3 for this lib.

[karlnapf]

I agree in principle! This has many nice advantages. In particular the fact that once the interface is fix, one can update/improve linalg without having to re-install Shogun.

However, I have some second thoughts as well.

One thing is that if we have a seperate lib, this one will depend on the Shogun core. In that way it will never be possible to use it as its own library. On the other hand, we to use it in Shogun as a hard requirement. This means that we have a two way dependency. So making it separate is merely a restructuring of the src directory ....

There is quite a bit of hassle involved in creating a build for both systems, with seperate cmake environments etc. On the other hand, the math library would be c++ only, so not many builds added.

I summary, I am pro, but advise some care when doing this. Also, if this transition is done, it should be done in a very short time window to avoid all sorts of legacy artefacts. @lambday @besser82 if you want to lead this, I would be happy to help.

But also curious what @vigsterkr says

[besser82]

@lambday Thank you very much! ^^ About the plugin-stuff: I think you got me wrong with the plugin thing… THe current linalg-thing won't be a plugin of Shogun… It will be a lib interfaced by Shogun. The plugin-stuff in this context is more like: Try to generalize / abstract the operations performed using Eigen or ViennaCL (and possibly other backends that may be there like ATLAS / BLAS(?)) and put them into plugins interfaced by our separated linalg-thing.

[besser82]

@karlnapf What in Shogun core is so specific, we cannot split into the new lib? I mean: There's always a way to break inter-circular dependencies…

I can do the CMake-part, work along on the code changes neccessary and guide the people involved with code-changes and all…

[lambday]

@besser82 ah I got your point. Maybe we should discuss in some more details about the design if we really decide to go forward for it.

If I didn't get this wrong, I think what @karlnapf means is that the linalg lib is supposed to provide API for SG-types. So that makes it dependent on Shogun core. Maybe there are ways to make it work with

• more generic Vector, Matrix (with some assumption about the members of those types, i.e. data(), size(), etc etc), or
• Put matrix/vector types in math lib, now Shogun core is a client of that math lib

@karlnapf please let me know if this is what you're concerned about.

Really want to hear @vigsterkr and @lisitsyn s' thoughts on this.

[karlnapf]

Yes the connection to the SGTypes is what I meant. I am just afraid that we start doing this and then get stuck somewhere in the middle. In my eyes, we should focus on actually integrating linalg first, and building the plugin framework first. Once both of those things are done, we can put it in a separate lib. Having said that, we can obviously already start drafting something. But I would be careful to open another big construction side at this point in time -- so many other things going on. Apart from that, I am all up for it.

Let's focus on integrating/finishing linalg first:

• Make use of it in Shogun code
• Put in operations that are desperately needed (cholesky, inverse)
• Make sure the GPU integration works and is used in Shogun
• Remove NATIVE backend ....
• etc etc