atztogo
commented
3 days ago I think, probably, we can avoid calling LAPACKE routines in the C code in phono3py. There are two types of calling those routines:
- Calling a function that blocks calculation (diagonalization, pinv).
- Many small callings of a function that are expected running in parallel. In this case, single thread BLAS should be used because OpenMP is used to call the function concurrently in my implementation. If I remember correctly, this way of using is only the following part of diagonalization (
zheev) of dynamical matrices over many q-points: https://github.com/phonopy/phono3py/blob/59dc78e98238eba932612b1f05406f07e39c53b6/c/phonon.c#L228
For 1, calling the LAPACK routines via scipy is fine if the BLAS is multithreaded. In old days, installation of scipy was sometimes difficult, so I tried to avoid relying on it. But now I feel OK to depend on scipy, so except for experimental usage (https://phonopy.github.io/phono3py/direct-solution.html#solver-choice-for-diagonalization), use of LAPACKE may be avoided at least for the release version.
For 2, currently I have no idea how to call selectively the multithreaded and single thread BLASs in one installation of phono3py. So I usually choose multithreaded BLAS, and the zheev in the above case is called sequentially over q-points. This diagonalization can be replaced by using numpy eigh in python. I am not sure the performance calling numpy over many q-points (i.e., overhead of python and wrapping LAPACK in numpy), but if this part is realized as not a bottle neck of the phono3py calculation, we can avoid using LAPACKE.
Overall, it is a good challenge to avoid using LAPACKE in a way as written above. Simply I had no reason to try it.
LecrisUT
I think it is good to consider the new C++26 proposal for
<linalg>, specifically section 9.2It seems unlikely that LAPACK interface would be ported to standard library and when BLAS is ported to C++ std, I don't believe the BLAS provider projects would have much benefit. Even currently there are issues:
lapacke.his not packaged in all providerspkg-configsupportFindLAPACK.cmakedoes not actually add the relevantincludeto work properlyThe paper recommends other C++ native libraries like Armadillo, Eigen3, etc. It might be useful to look into how the support for these would look like. But in the meantime let's discuss a plan of action
Short-term
Continue to use
lapacke.hand test for the presence of the header, and fail the build if it's not present. The user would have to define themselves overrides likeBLA_VENDORto select the vendor that is compatible like OpenBLAS. There are various bugs that need to be addressed upstream:pkg-configfilesFindLAPACK.cmakeshould have an interface to override the include directoryWith regards to the packaged wheels, these will contain bundled BLAS and LAPACK implementations compatible with
cibuildwheel. How well this works, we will have to find out and work withcibuildwheelfolks. But hopefully the user can rely on thesdistbuilds to work on their arbitrary environments.Future
For the future support we should consider some other options:
lapackewrappersFetchContentin case the user does not have the packageIf it's possible to get some performance benchmarking or profiling that would be very helpful for guiding which approach is most suitable.