Bihaqo
commented
6 years ago On the other hand, the QR version seems slower..
Open Bihaqo opened 6 years ago
Bihaqo
commented
6 years ago On the other hand, the QR version seems slower..
Right now Frobenious norm has a flag "differentiable" which chooses between fast and stable QR version and differentiable naive version.
It seems to be easy to manually implement backward pass for the QR version: we need to orthogonalize the tensor in both directions and then for differentiating w.r.t. k-th core we can represent the norm as: ||A||^2_F = ||U_1 U_2 ... U_k-1 (L_k-1 G_k R_k) U_k+1 ... ||^2 = ||L_k-1 G_k R_k||^2
where L_k-1 and R_k doesn't depend on G_k.
If anyone lacks a stable and differentiable Frobenius norm, I'll implement.