Open rgommers opened 12 months ago
The relevant code is: https://github.com/numpy/numpy/blob/6207a5203983e720cd36f49a39ba8ea3433c3231/numpy/core/include/numpy/npy_common.h#L387-L392
@ashish-2022 would you mind providing a full build log and the build command you used? Also, this code has changed a lot in the main
branch, so if you could check if you are getting the same error there, that would be helpful.
The relevant code is:
@ashish-2022 would you mind providing a full build log and the build command you used? Also, this code has changed a lot in the
main
branch, so if you could check if you are getting the same error there, that would be helpful.
Ok I'll try to build from latest code on GitHUB
Complete build log(code taken from https://files.pythonhosted.org/packages/78/23/f78fd8311e0f710fe1d065d50b92ce0057fe877b8ed7fd41b28ad6865bfc/numpy-1.26.1.tar.gz)
Log Attached. numpy_1_26_1_build_log.txt
Command used to build:
python -m pip install .
Code from main branch exited much earlier on my setup at following check:
Has header "complex.h" : YES
Checking for type "_Fcomplex" : NO
..\numpy\_core\meson.build:143:4: ERROR: Problem encountered: "complex.h" header does not include complex type _Fcomplex
A full log can be found at D:\Test_Numpy\numpy\.mesonpy-gt7tmyk6\meson-logs\meson-log.txt
[end of output]
Full log here: build_log_main.txt
Hmm, I assume this happens due to https://github.com/numpy/numpy/blob/cdfbdf428d9df9c7119cecae323512a4cd3f57b7/numpy/_core/meson.build#L127 We should probably special-case Intel there, right?
Hmm, I assume this happens due to
We should probably special-case Intel there, right?
This can be achieved using compiler.get_id()
, this is the list: https://mesonbuild.com/Reference-tables.html#compiler-ids
@ashish-2022 Can you check whether you can build numpy with #25044?
Hi @lysnikolaou, I had tried just putting that condition in meson.build but we will need more changes related to intel compiler complex types.
[9/491] Compiling C++ object numpy/_core/libnpymath.a.p/src_npymath_halffloat.cpp.obj
FAILED: numpy/_core/libnpymath.a.p/src_npymath_halffloat.cpp.obj
"icl" "-Inumpy\_core\libnpymath.a.p" "-Inumpy\_core" "-I..\numpy\_core" "-Inumpy\_core\include" "-I..\numpy\_core\include" "-I..\numpy\_core\src\npymath" "-I..\numpy\_core\src\common" "-IC:\DE-Python\Include" "-ID:\Test_Numpy\numpy\.mesonpy-q6_9ehof\meson_cpu" "-DNDEBUG" "/MD" "/nologo" "/showIncludes" "/utf-8" "/W2" "/EHsc" "/std:c++17" "/permissive-" "/O3" "/arch:SSE3" "-DNPY_HAVE_SSE2" "-DNPY_HAVE_SSE" "-DNPY_HAVE_SSE3" "-DMS_WIN64=" "/Fdnumpy\_core\libnpymath.a.p\src_npymath_halffloat.cpp.pdb" /Fonumpy/_core/libnpymath.a.p/src_npymath_halffloat.cpp.obj "/c" ../numpy/_core/src/npymath/halffloat.cpp
icl: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '/Qdiag-disable:10441' to disable this message.
C:\Program Files (x86)\Intel\oneAPI\compiler\latest\windows\compiler\include\complex.h(37): warning #1224: #warning directive: "The /Qstd=c99 compilation option is required to enable C99 support for C programs"
#warning "The /Qstd=c99 compilation option is required to enable C99 support for C programs"
^
../numpy/_core/include/numpy/npy_common.h(397): error: expected a ";"
typedef double _Complex npy_cdouble;
See full log here: intel_icl_log.txt
Then I also included "/Qstd=c99" flag in compilation but there was no use. need to check with some intel expert @rgommers
Unfortunately I don't have much experience with Intel and also no Windows machine to reproduce this issue at the moment, so I'll leave this to someone else.
From commits history following members can assign this to right person: @mattip, @chaoyu3
As the intel compiler is proprietary, we would need someone with a proper license to work through this. I do not have such a license.
It looks like at least the old intel compiler doesn't like _Complex
in C++ (on windows?). My suspicion would be that needs to be tweaked, half of the branching necessary is already there. Can't we rely on a standard form in C++ mode in all cases?
In either case, it needs someone with the compiler to try out what works well here.
@mattip that is no longer the case. Well, it's proprietary still, but freely available - you can simple download and install it, no license required.
The installers are quite cumbersome indeed. You can deselect most components though, so it shouldn't take that much space. That said, this is still a bit of a time sink, and this isn't super urgent probably. I'm planning to get a Windows laptop at the end of the month to be able to deal with this kind of thing more easily. So I can circle back to it then - before 2.0 at least.
Somehow the installer works differently via a remote desktop, it seems to work better with direct access.
The latest oneAPI compiler suite I installed came with icx.exe
and not icl.exe
. I needed to do this to build:
<install path>\setvars.bat intel64
set CC=icx.exe
set CXX=icx.exe
pip install scipy-openblas64 -r build_requirements.txt -r test_requirements.txt
REM use scipy-openblas wheels to provide OpenBLAS
pip install scipy-openblas64
set PKG_CONFIG_PATH=<path>\.openblas
pip install . --no-build-isolation
On HEAD I get the same error as in the comment above in the compiler check stage. I am starting to work through the needed changes. Many of the warnings come from icx.exe
being a clang-based c++ compiler which is used to compile c code.
Hi @mattip, This is how I was building earlier version's of numpy 1.24.3 with intel MKL:
:: ------------------------------------------------------------------------------
:: Compile and install numpy
:: ------------------------------------------------------------------------------
:: Download numpy
powershell Invoke-WebRequest -Uri 'https://files.pythonhosted.org/packages/2c/d4/590ae7df5044465cc9fa2db152ae12468694d62d952b1528ecff328ef7fc/numpy-1.24.3.tar.gz' -OutFile 'numpy-1.24.3.tar.gz'
tar -xvzf numpy-1.24.3.tar.gz
set NUMPY_DIR="numpy-1.24.3"
:: Create site.cfg
echo [mkl] > %NUMPY_DIR%/site.cfg
echo include_dirs = C:\Program Files (x86)\Intel\oneAPI\mkl\latest\include >> %NUMPY_DIR%/site.cfg
echo library_dirs = C:\Program Files (x86)\Intel\oneAPI\mkl\latest\lib\intel64 >>%NUMPY_DIR%/site.cfg
echo libraries = mkl_rt >> %NUMPY_DIR%/site.cfg
:: Start build
python -m pip install --verbose %NUMPY_DIR%/
:: Copy intel MKL libs for numpy
mkdir "%PYTHONHOME%\Lib\site-packages\numpy\.libs"
copy "C:\Program Files (x86)\Intel\oneAPI\mkl\latest\redist\intel64\*.dll" %PYTHONHOME%\Lib\site-packages\numpy\.libs
robocopy "C:\Program Files (x86)\Intel\oneAPI\mkl\latest\redist\intel64\1033" %PYTHONHOME%\Lib\site-packages\numpy\.libs\1033
Please note it required to link to intel MKL libs from site-packages/numpy/.libs/
There was some code I had seen long back for this (dont remember where it is, may be in one of init.py)
Right, but we no longer use site.cfg
There are many warnings from CPython about a missing declaration of struct timeval
. This was fixed for CPython 3.12, so I would recommend using that for compilation with the oneAPI compiler (or clang-cl for that matter). See python/cpython#82863 and python/cpython#77532
There are also many warnings about "warning: explicit comparison with NaN in fast floating point mode". It seems meson should use at least -fhonor-infinities -fhonor-nans
according to this issue in another repo
We should be disabling -ffast-math
for that on Intel compilers. It's missing in NumPy; I did do it for SciPy:
https://github.com/scipy/scipy/blob/ae026f72fbc3b6c60a5d5b9d2e392b5c1919fd44/meson.build#L91-L117
OK. I will add that to #25044
I tried to compile with python3.13t (no GIL) , and got the same error.
Transferred from https://github.com/numpy/numpy/issues/22711#issuecomment-1785115279 (bug report by @ashish-2022).:
I get this error when compiling numpy 1.26.1 with intel MKL on Windows (Python 3.11.6). is there any extra environment variables I need to set for this?