search

Roger-luo / MAGMA.jl

A Julia binding for MAGMA, don't drink it
MIT License
10 stars 2 forks source link

Feature request: Batched routines #12

Open GiggleLiu opened 5 years ago

GiggleLiu commented 5 years ago

Especially batched svd. Batched routines will increase the performance significantly on GPU.

Want this feature desperately :smile:

Cool project, wondering is there a milestone for v0.1? We'd like to use MAGMA as our SVD backend in our project about tensor network algorithms.

JerryChen97 commented 5 years ago

I will develop the batched version ASAP.

GiggleLiu commented 5 years ago

Cool, many thanks. Another welcoming feature is randomized SVD. This algorithm has better performance on GPU. Do you have any plan about this?

A reference: https://epubs.siam.org/doi/10.1137/090771806

Roger-luo commented 5 years ago

I don't think it would be a good idea to mix other implementation with a binding package. It makes it hard to maintain, this package is going to be a dependency of CuArrays for SVD/QR and other implementation, and it is just a binding of MAGMA. We could implement these specific algorithm directly there (or even just in a separate package like what they do in pytorch)

And @JerryChen97 let's focus on your proposal first (there are a bunch of bindings need to be done first), and if you want to work on this, we could do it as an extension of your SoC project.

GiggleLiu commented 5 years ago

Sure, let it be an extension would be better.

I am not an expert. Wondering how difficult it is to batch an operation for MAGMA? Trivial, or must rewrite everything.

Roger-luo commented 5 years ago

It just requires cuda 9.0+ I believe. But I need to try them out after magma is ready.

Roger-luo commented 5 years ago

I think batched routines should be generic enough to implement with a similar approach here:

https://gist.github.com/Roger-luo/a7cddd8cf5902c5b43e09ea16acc74da

Or just use Hydra in principal. There's no need to write them one by one.