Symeig unstable in float precision

lucmos commented 5 years ago

🐛 Bug

symeig produces wrong eigenvalues on some matrix in torch.float precision, both on cuda and on cpu.

To Reproduce

I'm not sure about what causes the unstability on some matrices.

Steps to reproduce the behavior:

Clone this repository
Execute the notebook to reproduce the behaviour

You can see the output of the notebook here, where float precision gives negative eigenvalues.

Expected behavior

float precision and double precision should give similar results

Environment

PyTorch version: 1.2.0
Is debug build: No
CUDA used to build PyTorch: 10.0.130

OS: Manjaro Linux
GCC version: (GCC) 9.1.0
CMake version: version 3.15.1

Python version: 3.7
Is CUDA available: Yes
CUDA runtime version: 10.1.168
GPU models and configuration: GPU 0: GeForce RTX 2080 Ti
Nvidia driver version: 430.26
cuDNN version: /usr/lib/libcudnn.so.7.6.1

Versions of relevant libraries:
[pip3] numpy==1.16.4
[pip3] torch==1.2.0
[pip3] torch-cluster==1.4.2
[pip3] torch-geometric==1.3.0
[pip3] torch-scatter==1.3.1
[pip3] torch-sparse==0.4.0
[pip3] torch-spline-conv==1.1.0
[pip3] torchvision==0.4.0
[conda] Could not collect

Additional context

The matrix is (theoretically) known to have positive eigenvalues and the first one should be (close to) zero.

cc @vishwakftw @SsnL @jianyuh

vishwakftw commented 5 years ago

Thank you for raising the issue, @LucaMoschella.

The matrix appears to be ill-conditioned (which can be validated by the additional context that you have provided), in which cases single precision algorithms for computing eigenvalues aren't reliable.

However, if one where to compare the results of NumPy and PyTorch, it should be duly noted that the way NumPy computes the eigenvalues for single precision matrices is by converting it to double precision, performing the operation and then converting the result back to single precision. A relevant function used in SciPy is the scipy.linalg.lapack.find_best_lapack_type which determines the optimal type for a variety of linear algebra methods.

lezcano commented 2 years ago

Closing this issue as rhis is expected. In many iterative algorithms, double precision is necessary when dealing with singular matrices or matrices with a small eigengap, and there's not much we can do about that as it comes from the numerical properies of the algorithm.

pytorch / pytorch