Open michaelweinold opened 1 year ago
This affects the following sections in the documentation:
Due to licensing conflicts, recent versions of SciPy do not include UMFpack. UMFpack is faster than SuperLU, especially for repeated calculations. Python wrappers for UMFpack must be installed separately using scikits.umfpack.
Setup > macOS (Apple Silicon/ARM)
Brightway runs on the new Apple Silicon ARM architecture. However, the super-fast linear algebra software library pypardiso is not compatible with the ARM processor architecture. To avoid critical errors during instruction that would break core functionality, a different version of Brightway (brightway_nosolver) must be installed, which includes a different linear algebra software library (scikit-umfpack).
The documentation for Accelerate sparse matrices is here.
I also added a warning to let people know if they don't have a fast LA library.
Given that using Accelerate
would mean a lot of development work on a platform with has scikit-umfpack
already, this is a low priority.
Numpy 2.0.0
now (again) has Apple Accelerate support out-of-the-box.
So I think this may not be the case:
Given that using Accelerate would mean a lot of development work on a platform with has scikit-umfpack already, this is a low priority.
And this might potentially be interesting for:
I'll try to check if I can compare performance on Apple Silicon and post the results here.
Aside from performance, we could further simplify/unify the installation instructions.
Sure, I saw this too. We are currently pinning stuff to numpy<2 because we don't have enough testing to see if it works, but this looks promising.
Having done some testing, I quickly realized that the actual magic happens in the sp.sparse.linalg.spsolve(technosphere_matrix, demand_array)
function, which is called by bw2calc.lca_base.LCABase.solve_linear_system
, which is called by bw2calc.lca.LCA.lci_calculation
.
Unfortunately, the SciPy documentation (1.13.X) mentions that:
A recent version of OpenBLAS, MKL or ATLAS. The Accelerate BLAS library is no longer supported. \ - Scipy Toolchain Roadmap
An archived SciPy GitHub-wiki entry details the reasoning for this missing support.
There are some efforts:
However, it looks like this might still take a while 😢...
Ok, before @romainsacchi posts his findings here:
For some reason I did not find this previously: SciPy from 1.14.0
DOES now have Apple Accelerate support.
Installing SciPy such that is is actually built against Accelerate is not yet straightforward. Conda failed to do so when I tried... I had better luck with just pip (yeah, yeah, @cmutel 😜).
I found performance did improve for some operations - a quick testing notebook is on this branch:
a quick testing notebook is on this branch
Is it though?
scipy
in December 2018 dropped and in May 2021 re-introduced support for the Apple Accelerate library (compare also the SciPy roadmap and their GitHub repo wiki page).This library
It could therefore provide a highly optimized alternative to the current
bw25nosolver
workaround (compare the documentation).Investigation
scipy
installation has Accelerate supportscipy
with Accelerate supportImplementation
bw2calc
Compare @cmutel's existing notebook "Compare speed of iterative solvers".
Documentation
Installation
Sparse Solvers
Discussion on Hackernews