Open jcmgray opened 1 year ago
Thanks for the report @jcmgray. This looks suspiciously similar to https://github.com/numpy/numpy/issues/21950, which ended up being a gfortran
bug.
(Can't check with
blas=*=openblas
as this doesn't appear to work more generally)
That's a little odd. Either way, openblas
is the default so a simple mamba create -n tmp-env numpy=1.24.1
will use openblas
.
I can confirm the problem with netlib
and accelerate
, while openblas
works fine:
% mamba create -n numpy-arm64-issue python=3.11 numpy=1.24.1 # uses openblas
% mamba activate numpy-arm64-issue
% python eigh_issue.py
7.67097720664267e-16
7.703544722840589e-16
4.057796502112606e-15
% mamba install "libblas=*=*netlib"
% python eigh_issue.py
1.7417791256257247
1.741779125625725
7.5457859052319
% mamba install "libblas=*=*accelerate"
% python eigh_issue.py
1.7299060495984222
1.7299060495984222
7.154916026621027
% mamba install "libblas=*=*openblas" # works fine
There are other test failures. Here is the numpy.linalg
test suite result for accelerate
:
FAILED tests/test_linalg.py::TestSVDHermitian::test_herm_cases - AssertionError: In test case: <LinalgCase: hcsingle>
FAILED tests/test_linalg.py::TestPinvHermitian::test_herm_cases - AssertionError: In test case: <LinalgCase: hcsingle>
FAILED tests/test_linalg.py::TestEighCases::test_herm_cases - AssertionError: In test case: <LinalgCase: hcsingle>
3 failed, 383 passed, 2 skipped, 27 deselected, 2 xfailed in 1.63s
And for netlib
:
FAILED tests/test_linalg.py::TestSVDHermitian::test_herm_cases - AssertionError: In test case: <LinalgCase: hcsingle>
FAILED tests/test_linalg.py::TestPinvHermitian::test_herm_cases - AssertionError: In test case: <LinalgCase: hcsingle>
FAILED tests/test_linalg.py::TestEighCases::test_herm_cases - AssertionError: In test case: <LinalgCase: hcsingle>
FAILED tests/test_linalg.py::TestCholesky::test_basic_property[complex64-shape1] - numpy.linalg.LinAlgError: Matrix is not positive definite
FAILED tests/test_linalg.py::TestCholesky::test_basic_property[complex64-shape2] - numpy.linalg.LinAlgError: Matrix is not positive definite
FAILED tests/test_linalg.py::TestCholesky::test_basic_property[complex64-shape3] - numpy.linalg.LinAlgError: Matrix is not positive definite
FAILED tests/test_linalg.py::TestCholesky::test_basic_property[complex64-shape4] - numpy.linalg.LinAlgError: Matrix is not positive definite
FAILED tests/test_linalg.py::TestCholesky::test_basic_property[complex128-shape1] - numpy.linalg.LinAlgError: Matrix is not positive definite
FAILED tests/test_linalg.py::TestCholesky::test_basic_property[complex128-shape2] - numpy.linalg.LinAlgError: Matrix is not positive definite
FAILED tests/test_linalg.py::TestCholesky::test_basic_property[complex128-shape3] - numpy.linalg.LinAlgError: Matrix is not positive definite
FAILED tests/test_linalg.py::TestCholesky::test_basic_property[complex128-shape4] - numpy.linalg.LinAlgError: Matrix is not positive definite
11 failed, 375 passed, 2 skipped, 27 deselected, 2 xfailed in 1.93s
With openblas
, all tests pass.
That's a little odd. Either way, openblas is the default so a simple mamba create -n tmp-env numpy=1.24.1 will use openblas.
Yes likely a separate issue, but thought I'd note that at least on my Mac the default install (i.e, this exact command) is the netlib
blas variant, not openblas
. When I use openblas I get an error that is essentially libcblas.3.dylib' (no such file)
.. perhaps I am missing some dep that makes conda/mamba avoid openblas.
Hmm, no idea why. That's a significant performance penalty, so I'd consider getting netlib
by default a bug. The openblas
packages for both x86-64 and arm64 at https://anaconda.org/conda-forge/openblas are up to date, so I can't explain what is happening there. Maybe someone more familiar with the BLAS selection mechanism can.
Seems like I need to get the blas testing series up to speed for 1.24.
Yes likely a separate issue, but thought I'd note that at least on my Mac the default install (i.e, this exact command) is the
netlib
blas variant, notopenblas
. When I use openblas I get an error that is essentiallylibcblas.3.dylib' (no such file)
.. perhaps I am missing some dep that makes conda/mamba avoid openblas.
openblas should be the default also on osx, and in any case it should definitely work. However, you're mixing channels (default and conda-forge), which is a very likely cause for these problems. Could you please try
conda config --add channels conda-forge
conda config --set channel_priority strict
and then either a new environment, or using conda update --all
?
openblas should be the default also on osx, and in any case it should definitely work. However, you're mixing channels (default and conda-forge), which is a very likely cause for these problems. Could you please try
Ah sorry yes you are right, thanks for the suggestion.
I am using a Mac mini M1 and recently found this issue. It works fine with "libblas=*=*openblas"
and "libblas=*=*netlib"
. However, the eigh issue appears when I update Numpy using "libblas=*=*accelerate"
. Probably this is because they didn't update the libblas 3.9.0-16_osxarm64_accelerate
in conda-forge? @jcmgray is your problem solved? Where can I ask for updating the libblas 3.9.0-16_osxarm64_accelerate
package since complex eigenvector problems are significant in physics?
numpy/numpy#24640
Experienced the same problem on numpy 1.26.0 with accelerate blas.
Solution to issue cannot be found in the documentation.
Issue
On osx-arm64 with numpy==1.24 installed from conda-forge,
np.linalg.eigh
produces incorrect results for complex data types:Other linalg such as svd appears to be working (though I haven't checked exhaustively).
I have checked this with:
blas=*=acclerate
blas=*=netlib
(Can't check with
blas=*=openblas
as this doesn't appear to work more generally)Using
numpy
from pypi the code is working fine. Thus raising the issue here assuming it is something to do with the linked lapack, but apologies if this should be in the mainnumpy
repo.Possibly related to #21950, thought the issue here is not simply normalization of the eigenvectors.
Installed packages
Environment info