search

conda-forge / pythran-feedstock

A conda-smithy repository for pythran.
BSD 3-Clause "New" or "Revised" License
1 stars 11 forks source link

ARM OSX Migrator #48

Closed h-vetinari closed 3 years ago

h-vetinari commented 3 years ago

Validate #47 by adding artefact persistence for local testing

conda-forge-linter commented 3 years ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

h-vetinari commented 3 years ago

Gentle ping @xhochy to kindly test the osx-arm artefacts here before this is merged (previously #47) .

xhochy commented 3 years ago

Fails with:

+ python -c 'import dprod'
+ pythran -v simple_numexpr.py -DUSE_XSIMD -march=native
INFO:      sys file exists: /Users/uwe/mambaforge/conda-bld/pythran_1623267118909/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/python3.8/site-packages/pythran/pythran.cfg
INFO: platform file exists: /Users/uwe/mambaforge/conda-bld/pythran_1623267118909/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/python3.8/site-packages/pythran/pythran-darwin.cfg
WARNING:     user file does not exist: /Users/uwe/.pythranrc
INFO: pythranrc section [pythran] is valid and options are correct
INFO: pythranrc section [typing] is valid and options are correct
INFO: pythranrc section [compiler] is valid and options are correct
clang-11: warning: -Wl,-export_dynamic: 'linker' input unused [-Wunused-command-line-argument]
/var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/tmpgsxgb3g2/source.c:1:10: fatal error: 'cblas.h' file not found
#include <cblas.h>
         ^~~~~~~~~
1 error generated.
running build_ext
new_compiler returns <class 'distutils.unixccompiler.UnixCCompiler'>
building 'simple_numexpr' extension
C compiler: arm64-apple-darwin20.0.0-clang++ -fno-strict-aliasing -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O3 -Wall -arch arm64 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O3 -pipe -fdebug-prefix-map=${SRC_DIR}=/usr/local/src/conda/${PKG_NAME}-${PKG_VERSION} -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -flto -Wl,-export_dynamic -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem $PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/pythran-0.9.11 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -D_FORTIFY_SOURCE=2 -isystem $PREFIX/include -mmacosx-version-min=11.0

creating /var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/tmp0yupwm2y/var
creating /var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/tmp0yupwm2y/var/folders
creating /var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/tmp0yupwm2y/var/folders/km
creating /var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/tmp0yupwm2y/var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn
creating /var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/tmp0yupwm2y/var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T
compile options: '-DENABLE_PYTHON_MODULE -D__PYTHRAN__=3 -DUSE_XSIMD -DPYTHRAN_BLAS_BLAS -I$PREFIX/lib/python3.8/site-packages/pythran -I$PREFIX/lib/python3.8/site-packages/numpy/core/include -I$PREFIX/include -I$PREFIX/include/python3.8 -c'
extra options: '-std=c++11 -fno-math-errno -w -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -std=c++14 -fmessage-length=0 -isystem $PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/pythran-0.9.11 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -march=native'
arm64-apple-darwin20.0.0-clang++: /var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/tmp2d8qjsd6.cpp
clang-11: error: the clang compiler does not support '-march=native'
WARNING: Compilation error, trying hard to find its origin...
WARNING: Nop, I'm going to flood you with C++ errors!
CRITICAL: Cover me Jack. Jack? Jaaaaack!!!!
E: error: Command "arm64-apple-darwin20.0.0-clang++ -fno-strict-aliasing -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O3 -Wall -arch arm64 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O3 -pipe -fdebug-prefix-map=${SRC_DIR}=/usr/local/src/conda/${PKG_NAME}-${PKG_VERSION} -fdebug-prefix-map=/Users/uwe/mambaforge/conda-bld/pythran_1623267118909/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold=/usr/local/src/conda-prefix -flto -Wl,-export_dynamic -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/uwe/mambaforge/conda-bld/pythran_1623267118909/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/include -fdebug-prefix-map=/Users/uwe/mambaforge/conda-bld/pythran_1623267118909/test_tmp=/usr/local/src/conda/pythran-0.9.11 -fdebug-prefix-map=/Users/uwe/mambaforge/conda-bld/pythran_1623267118909/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold=/usr/local/src/conda-prefix -D_FORTIFY_SOURCE=2 -isystem /Users/uwe/mambaforge/conda-bld/pythran_1623267118909/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/include -mmacosx-version-min=11.0 -DENABLE_PYTHON_MODULE -D__PYTHRAN__=3 -DUSE_XSIMD -DPYTHRAN_BLAS_BLAS -I/Users/uwe/mambaforge/conda-bld/pythran_1623267118909/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/python3.8/site-packages/pythran -I/Users/uwe/mambaforge/conda-bld/pythran_1623267118909/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/python3.8/site-packages/numpy/core/include -I/Users/uwe/mambaforge/conda-bld/pythran_1623267118909/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/include -I/Users/uwe/mambaforge/conda-bld/pythran_1623267118909/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/include/python3.8 -c /var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/tmp2d8qjsd6.cpp -o /var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/tmp0yupwm2y/var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/tmp2d8qjsd6.o -std=c++11 -fno-math-errno -w -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -std=c++14 -fmessage-length=0 -isystem /Users/uwe/mambaforge/conda-bld/pythran_1623267118909/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/include -fdebug-prefix-map=/Users/uwe/mambaforge/conda-bld/pythran_1623267118909/test_tmp=/usr/local/src/conda/pythran-0.9.11 -fdebug-prefix-map=/Users/uwe/mambaforge/conda-bld/pythran_1623267118909/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold=/usr/local/src/conda-prefix -march=native" failed with exit status 1
h-vetinari commented 3 years ago

@serge-sans-paille Does pythran host-depend on libcblas or is that just necessary for the tests? (numpy does host-depend on libcblas, but that's not transitive)

serge-sans-paille commented 3 years ago

the cblas error shows up but it's non blocking and I have an issue opened on it https://github.com/serge-sans-paille/pythran/issues/1798 The real issue is clang-11: error: the clang compiler does not support '-march=native' you can just remove the '-march=native' flag os OSX

h-vetinari commented 3 years ago

Thanks @serge-sans-paille for the context - I had seen that error but wanted to wait for more info about cblas before pushing.

Also, thanks a lot @xhochy for your help with this! 😊

h-vetinari commented 3 years ago

@xhochy - whenever you're ready, this could be taken for another spin 🙃

xhochy commented 3 years ago

It did fail again but I did take the chance and committed the potential fix myself. I'll have to retest tomorrow.

h-vetinari commented 3 years ago

Thanks! I specifically looked up what arch would show up on an M1, but it seems that I was wrong somehow - sorry

xhochy commented 3 years ago

Selectors are different from the arch. If you have the arch, you could do target_platform == "osx-arm64".

xhochy commented 3 years ago

This works fine now!

h-vetinari commented 3 years ago

Selectors are different from the arch. If you have the arch, you could do target_platform == "osx-arm64".

I get that, but I thought that if you're actually executing on an M1 it wouldn't be the target_platform anymore... 🤔

This works fine now!

Great to hear, thanks a lot!

@serge-sans-paille, any opposition to merging it like this?

h-vetinari commented 3 years ago

@serge-sans-paille, any opposition to merging it like this?

Give me a sec, I need a push to remove the artefact persistence

xhochy commented 3 years ago

target_platform is independent of the build system, only build_platform changes between CI with cross-compiling and building directly on the M1.

h-vetinari commented 3 years ago

Who wants to do the honour of clicking the merge button? 😅

h-vetinari commented 3 years ago

Ping for merge :) @conda-forge/pythran