Feature request: Singular-value decomposition (SVD)

[nsthorat]

From @caisq on January 20, 2018 17:16

This can mimic tensorflow's otf.linalg.svd: https://www.tensorflow.org/api_docs/python/tf/svd

This is useful for, among other things, the generating of orthogonal initial weights for RNNs.

Copied from original issue: tensorflow/tfjs-core#571

[kedevked]

Hi, I will work on it

[caisq]

Thanks, @kedevked Looking forward to your PR. Let us know if you need any help or guidance.

[kedevked]

I am going to implement it using the qr method. The algorithm is described here. I wonder from which discrepancy can we consider the convergence to be reached for a singular value, meaning that if at the ith iteration we have the value j and at the (i+1)th iteration we have the value k, I want to stop the iteration when k-i < threshold. What should the threshold be ? Also, @caisq , @nsthorat could you please review the solve PR, as I am going to use it to find the singular vector for each singular value ?

[ken-okabe]

I have a question, for node.

For Python version, svd is already there; https://www.tensorflow.org/api_docs/python/tf/linalg/svd and I guess these Python apis are based on C binaries.

tfjs for node is also based on the same C binary, I've heard. So I think it should be trivial to implement all of the apis that are already implemented in Python version, for tfjs for node.

I understand all of the apis of tfjs on browser is pure js and it seems a long way development to cover basic apis.

So, here's my question.

Is there any development policies that apis for both browser and node are synchlonized? If so, even there are exsiting binaries to bind from node, we must wait every api for browser pure js implementation.

If I miss something, plese advice. Thanks.

[lp74]

@kedevked, with a shift the convergence would be faster.

But I think that this problem is too way complex to be solved efficiently with a simple routine.

Tfjs needs to compute svd efficiently I guess.

Meanwhile, I use other libraries to compute svd and other useful stuffs, because I prefer to use efficiently the CPU than use the GPU in the wrong way.

See Lapack book: https://www.netlib.org/lapack/lug/node53.html

[kedevked]

@nsthorat @caisq can you please tell us if it is still in the roadmap of tfjs to implement linear algebra operators ? If so, maybe you can give some pointers in the direction to go. Thanks !

[EddieOne]

It would be nice to have a way to test SVD in TFJS.

[kedevked]

@EddieOne Yes indeed, that would be nice to have. But I guess the tfjs team for now have decided not to include this operators in the core library.

[EddieOne]

As I understand it, we need more floating points in JS to get the accuracy needed for SVD. But I am fuzzy on the details. I was going to try more interpolation without SVD and see what happens.

[tsol]

Not a single one implementation of SVD i found in NPM decomposes [8,9] matrix exactly like pyhons Numpy.lialg.svd function. I really hoped tfjs would have one...

[gaikwadrahul8]

Hi, @nsthorat

Thank you for opening this issue for tracking purposes. Since this issue has been open for a long time, the code/debug information for this issue may not be relevant with the current state of the code base.

The TFJs team is constantly improving the framework by fixing bugs and adding new features. We suggest you try the latest TFJs version with the latest compatible hardware configuration which could potentially resolve the issue. We can keep the issue open if it is still relevant. Please confirm if we need to keep the issue open.

Thank you for your support and cooperation.