Open meow464 opened 2 years ago
The flags -lflang -lflangrti -lpgmath -lompstub -lm -lpthread -lSystem
are the default linker flags for flang.
Problems:
-L/usr/local/flang-iOS/lib
but the flang libs should be probably somewhere under dist/
. I did try copying them to dist/hostpython3/lib/
and dist/lib
but it didn't work.It seems the numpy setup.py or cython were trying to build a test binary but the linker was not receiving /usr/local/flang-iOS
or dist/hostpython3/lib
. Adding an -L flag solved this.
Now I'm getting a duplicated symbol _xerbla_
in the files: libnumpy.a (_multiarray_umath.cpython-39-darwin.so.o)
and libnumpy.a (lapack_lite.cpython-39-darwin.so.o)
. But lapack_lite shouldn't be build if openblas is available? Or should it?
I think I made progress:
But I don't know if just commenting out the source of _xerbla_
like that is the right approach? Ideally numpy should detect openblas and disable lapack_lite, right? (I think _xerbla_
comes from lapack_lite).
In any case numpy still can't find openblas when deployed to a phone.
It seems numpy can find openblas but it shows the host's path /Users/tuco/Projects/kivy-OpenBLAS/dist/hostpython3/lib
. I'm able to run some examples but I still don't know for sure if it can use openblas.
I don't yet have a recipe, I just drop the flang and openblas .a files under dist/hostpython3/lib
and dist/lib
.
If I just do that and run the project the benchmark finishes in ~1.5 seconds but if I drop the .a files in xcode the run time goes up to ~5.5 seconds. Very weird.
I tried with pure numpy, size=1536
and it took ~8 seconds. (4096 took over 20 minutes and didn't finish).
The 5 seconds result from above was openblas and the 1.5 second must have been Accelerate.
Accelerate is included in the default kivy-ios project and even if numpy doesn't support it some stuff works and the symbols are there.
During the last ~2 months I got to what seems to be a working port of flang for iOS, but I still have to figure out how to make it work with numpy and kivy.
Here are the steps to get where I'm at, first build flang and openblas:
Make a kivy project:
mkdir kivy-OpenBLAS && cd kivy-OpenBLAS
python -m venv venv && . venv/bin/activate
git clone https://github.com/kivy/kivy-ios.git && pip install -e kivy-ios
You can make modification in the
kivy-ios/
folder and it will affect the installed kivy-ios.Create a new project as per usual:
toolchain build python3 kivy
Copy the openblas files to `dist/hostpython3/:
INSTALL/lib/libopenblas_armv8p-r0.3.19.dev.a
from the openblas folder todist/hostpython3/lib/libopenblas.a
. Note that you need to rename it.INSTALL/include
todist/hostpython3/include
So far openblas is only built for iOS therefore you will need to use
toolchain build numpy --arch arm64
to test numpy. To make changes to the numpy recipe simply editkivy-ios/kivy_ios/recipes/numpy/__init__.py
.My currently not working numpy recipe
``` from kivy_ios.toolchain import CythonRecipe from os.path import join import sh import shutil class NumpyRecipe(CythonRecipe): version = "1.20.2" url = "https://pypi.python.org/packages/source/n/numpy/numpy-{version}.zip" library = "libnumpy.a" libraries = ["libnpymath.a", "libnpyrandom.a"] include_dir = "numpy/core/include" depends = ["python"] hostpython_prerequisites = ["Cython"] cythonize = False def prebuild_arch(self, arch): if self.has_marker("patched"): return self.apply_patch("duplicated_symbols.patch") self.set_marker("patched") def get_recipe_env(self, arch): env = super().get_recipe_env(arch) # CC must have the CFLAGS with arm arch, because numpy tries first to # compile and execute an empty C to see if the compiler works. This is # obviously not working when crosscompiling env["CC"] = "{} {}".format(env["CC"], env["CFLAGS"]) env["FC"] = "/ur/local/bin/flang -L/usr/local/flang-iOS" # Disable Accelerate.framework by disabling the optimized BLAS and LAPACK libraries cause it's now unsupported env["NPY_BLAS_ORDER"] = "openblas" env["NPY_LAPACK_ORDER"] = "openblas" return env def build_arch(self, arch): super().build_arch(arch) sh.cp(sh.glob(join(self.build_dir, "build", "temp.*", "libnpy*.a")), self.build_dir) def reduce_python_package(self): dest_dir = join(self.ctx.site_packages_dir, "numpy") shutil.rmtree(join(dest_dir, "core", "include")) shutil.rmtree(join(dest_dir, "core", "tests")) shutil.rmtree(join(dest_dir, "distutils")) shutil.rmtree(join(dest_dir, "doc")) shutil.rmtree(join(dest_dir, "f2py", "tests")) shutil.rmtree(join(dest_dir, "fft", "tests")) shutil.rmtree(join(dest_dir, "lib", "tests")) shutil.rmtree(join(dest_dir, "linalg", "tests")) shutil.rmtree(join(dest_dir, "ma", "tests")) shutil.rmtree(join(dest_dir, "matrixlib", "tests")) shutil.rmtree(join(dest_dir, "polynomial", "tests")) shutil.rmtree(join(dest_dir, "random", "tests")) shutil.rmtree(join(dest_dir, "tests")) recipe = NumpyRecipe() ```All input is welcome!