search

IntelPython / dpctl

Python SYCL bindings and SYCL-based Python Array API library
https://intelpython.github.io/dpctl/
Apache License 2.0
99 stars 30 forks source link

Unable to build dpctl due to compilation errors in cython generated files #1448

Closed diptorupd closed 9 months ago

diptorupd commented 12 months ago

I tried building dpctl master from source and ran into several issues when icx tries to compile the Cython generated C++ files. I have pasted my conda environment below. Is there a certain version of cython that I should be using?

An example of the type of errors I am getting:


Error compiling Cython file:
------------------------------------------------------------
...
        if (ERef is NULL):
            raise ValueError("SyclEvent copy failed.")
        return pycapsule.PyCapsule_New(
            <void *>ERef,
            "SyclEventRef",
            &_event_capsule_deleter
            ^
------------------------------------------------------------

dpctl/_sycl_event.pyx:273:12: Cannot assign type 'void (*)(object) except *' to 'PyCapsule_Destructor'

Error compiling Cython file:

cimport cython.array
        ^
------------------------------------------------------------

dpctl/program/_program.pyx:28:8: 'cython.array' is not a valid cython.* module. Instead, use 'import cython' and then 'cython.array'.

My conda environment is:

#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
_openmp_mutex             5.1                       1_gnu  
_sysroot_linux-64_curr_repodata_hack 3                   haa98f57_10  
bandit                    1.7.5                    pypi_0    pypi
binutils_impl_linux-64    2.38                 h2a08ee3_1  
black                     23.3.0                   pypi_0    pypi
blas                      1.0                         mkl  
boltons                   23.0.0          py310h06a4308_0  
brotlipy                  0.7.0           py310h7f8727e_1002  
bzip2                     1.0.8                h7b6447c_0  
c-ares                    1.19.1               h5eee18b_0  
ca-certificates           2023.08.22           h06a4308_0  
certifi                   2023.7.22       py310h06a4308_0  
cffi                      1.15.1          py310h5eee18b_3  
cfgv                      3.3.1                    pypi_0    pypi
charset-normalizer        2.0.4              pyhd3eb1b0_0  
click                     8.1.3                    pypi_0    pypi
cmake                     3.26.4               h96355d8_0  
colorama                  0.4.6           py310h06a4308_0  
conda                     23.9.0          py310h06a4308_0  
conda-package-handling    2.2.0           py310h06a4308_0  
conda-package-streaming   0.9.0           py310h06a4308_0  
coverage                  7.2.2           py310h5eee18b_0  
cryptography              41.0.3          py310hdda0065_0  
cxxfilt                   0.3.0                    pypi_0    pypi
cython                    3.0.0           py310h5eee18b_0  
distlib                   0.3.6                    pypi_0    pypi
distro                    1.5.0              pyhd3eb1b0_1  
dpcpp-cpp-rt              2023.2.0            intel_49495    intel
dpcpp-llvm-spirv          2023.0.0            py310_25370    dppy/label/dev
dpcpp_impl_linux-64       2023.2.0            intel_49495    intel
dpcpp_linux-64            2023.2.0            intel_49495    intel
exceptiongroup            1.0.4           py310h06a4308_0  
expat                     2.5.0                h6a678d5_0  
filelock                  3.12.0                   pypi_0    pypi
flake8                    6.0.0           py310h06a4308_0  
gcc_impl_linux-64         11.2.0               h1234567_1  
gitdb                     4.0.10                   pypi_0    pypi
gitpython                 3.1.31                   pypi_0    pypi
gxx_impl_linux-64         11.2.0               h1234567_1  
icc_rt                    2023.2.0            intel_49495    intel
identify                  2.5.24                   pypi_0    pypi
idna                      3.4             py310h06a4308_0  
importlib-metadata        6.0.0           py310h06a4308_0  
iniconfig                 1.1.1              pyhd3eb1b0_0  
intel-cmplr-lib-rt        2023.2.0            intel_49495    intel
intel-cmplr-lic-rt        2023.2.0            intel_49495    intel
intel-opencl-rt           2023.2.0            intel_49495    intel
intel-openmp              2023.2.0            intel_49495    intel
isort                     5.12.0                   pypi_0    pypi
jsonpatch                 1.32               pyhd3eb1b0_0  
jsonpointer               2.1                pyhd3eb1b0_0  
kernel-headers_linux-64   3.10.0              h57e8cba_10  
krb5                      1.20.1               h143b758_1  
ld_impl_linux-64          2.38                 h1181459_1  
libcurl                   8.4.0                h251f7ec_0  
libedit                   3.1.20221030         h5eee18b_0  
libev                     4.33                 h7f8727e_1  
libffi                    3.4.4                h6a678d5_0  
libgcc-devel_linux-64     11.2.0               h1234567_1  
libgcc-ng                 11.2.0               h1234567_1  
libgomp                   11.2.0               h1234567_1  
libllvm14                 14.0.6               hdb19cb5_3  
libnghttp2                1.57.0               h2d74bed_0  
libssh2                   1.10.0               hdbd6064_2  
libstdcxx-devel_linux-64  11.2.0               h1234567_1  
libstdcxx-ng              11.2.0               h1234567_1  
libuuid                   1.41.5               h5eee18b_0  
libuv                     1.44.2               h5eee18b_0  
llvmlite                  0.41.0                   pypi_0    pypi
lz4-c                     1.9.4                h6a678d5_0  
mando                     0.7.1                    pypi_0    pypi
markdown-it-py            2.2.0                    pypi_0    pypi
mccabe                    0.7.0              pyhd3eb1b0_0  
mdurl                     0.1.2                    pypi_0    pypi
mkl                       2023.1.0         h213fc3f_46343  
mkl-service               2.4.0           py310h5eee18b_1  
mkl_fft                   1.3.8           py310h5eee18b_0  
mkl_random                1.2.4           py310hdb19cb5_0  
mypy-extensions           1.0.0                    pypi_0    pypi
ncurses                   6.4                  h6a678d5_0  
ninja                     1.10.2               h06a4308_5  
ninja-base                1.10.2               hd09550d_5  
nodeenv                   1.8.0                    pypi_0    pypi
numba                     0.58.0+0.g54c0cf862.dirty          pypi_0    pypi
numba-dpex                0.22.0.dev0+45.g48066fcee.dirty           dev_0    <develop>
numpy                     1.24.3          py310h5f9d8c6_1  
numpy-base                1.24.3          py310hb5e798b_1  
openssl                   3.0.11               h7f8727e_2  
packaging                 23.1            py310h06a4308_0  
pathspec                  0.11.1                   pypi_0    pypi
pbr                       5.11.1                   pypi_0    pypi
pip                       23.3            py310h06a4308_0  
platformdirs              3.5.1                    pypi_0    pypi
pluggy                    1.0.0           py310h06a4308_1  
pre-commit                3.3.2                    pypi_0    pypi
pycodestyle               2.10.0          py310h06a4308_0  
pycosat                   0.6.6           py310h5eee18b_0  
pycparser                 2.21               pyhd3eb1b0_0  
pyflakes                  3.0.1           py310h06a4308_0  
pyopenssl                 23.2.0          py310h06a4308_0  
pysocks                   1.7.1           py310h06a4308_0  
pytest                    7.3.1              pyhd8ed1ab_0    conda-forge
pytest-cov                4.1.0              pyhd8ed1ab_0    conda-forge
python                    3.10.13              h955ad1f_0  
pyyaml                    6.0                      pypi_0    pypi
radon                     6.0.1                    pypi_0    pypi
readline                  8.2                  h5eee18b_0  
requests                  2.31.0          py310h06a4308_0  
rhash                     1.4.3                hdbd6064_0  
rich                      13.4.1                   pypi_0    pypi
ruamel.yaml               0.17.21         py310h5eee18b_0  
ruamel.yaml.clib          0.2.6           py310h5eee18b_1  
scikit-build              0.17.6             pyh4af843d_0    conda-forge
setuptools                68.0.0          py310h06a4308_0  
six                       1.16.0                   pypi_0    pypi
smmap                     5.0.0                    pypi_0    pypi
spirv-tools               2021.4               hd09550d_0  
sqlite                    3.41.2               h5eee18b_0  
stevedore                 5.1.0                    pypi_0    pypi
sysroot_linux-64          2.17                h57e8cba_10  
tbb                       2021.8.0             hdb19cb5_0  
tk                        8.6.12               h1ccaba5_0  
toml                      0.10.2             pyhd3eb1b0_0  
tomli                     2.0.1           py310h06a4308_0  
tqdm                      4.65.0          py310h2f386ee_0  
truststore                0.8.0           py310h06a4308_0  
typing-extensions         4.7.1           py310h06a4308_0  
typing_extensions         4.7.1           py310h06a4308_0  
tzdata                    2023c                h04d1e81_0  
urllib3                   1.26.16         py310h06a4308_0  
virtualenv                20.23.0                  pypi_0    pypi
wheel                     0.41.2          py310h06a4308_0  
xz                        5.4.2                h5eee18b_0  
zipp                      3.11.0          py310h06a4308_0  
zlib                      1.2.13               h5eee18b_0  
zstandard                 0.19.0          py310h5eee18b_0  
zstd                      1.5.5                hc292b87_0
diptorupd commented 12 months ago

I also saw few warning:

warning: dpctl/_sycl_queue.pxd:32:57: The keyword 'nogil' should appear at the end of the function signature line. Placing it before 'except' or 'noexcept' will be disallowed in a future version of Cython.
warning: dpctl/_sycl_queue.pyx:124:61: The keyword 'nogil' should appear at the end of the function signature line. Placing it before 'except' or 'noexcept' will be disallowed in a future version of Cython.
oleksandr-pavlyk commented 12 months ago

I believe the error results from unrefreshed generated CXX files. Please delete _skbuild folder and rerun the build afresh. Please report if the build error persists.

oleksandr-pavlyk commented 11 months ago

@diptorupd were you able to resolve this?

oleksandr-pavlyk commented 10 months ago

@diptorupd ping

diptorupd commented 10 months ago

@oleksandr-pavlyk pong. I apologize for the delay in getting back.

I followed your advise and retried it on a clean environment on the master branch with dpcpp 2023.1. I got much further along than before, but then encountered:


[122/211] Building CXX object dpctl/tensor/CMakeFiles/_tensor_impl.dir/libtensor/source/copy_and_cast_usm_to_usm.cpp.o
warning: overriding currently unsupported rounding mode on this target [-Wunsupported-floating-point-opt]
warning: overriding currently unsupported use of floating point exceptions on this target [-Wunsupported-floating-point-opt]
2 warnings generated.
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "/home/diptorupd/miniconda3/envs/dpctl-devel/lib/python3.10/site-packages/skbuild/setuptools_wrap.py", line 645, in setup
    cmkr.make(make_args, install_target=cmake_install_target, env=env)
  File "/home/diptorupd/miniconda3/envs/dpctl-devel/lib/python3.10/site-packages/skbuild/cmaker.py", line 680, in make
    self.make_impl(clargs=clargs, config=config, source_dir=source_dir, install_target=install_target, env=env)
  File "/home/diptorupd/miniconda3/envs/dpctl-devel/lib/python3.10/site-packages/skbuild/cmaker.py", line 704, in make_impl
    raise SKBuildError(

An error occurred while building with CMake.
  Command:
    cmake --build . --target install --config Release --
  Install target:
    install
  Source directory:
    /home/diptorupd/Desktop/devel/dpctl
  Working directory:
    /home/diptorupd/Desktop/devel/dpctl/_skbuild/linux-x86_64-3.10/cmake-build

My conda environment is:

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
_openmp_mutex             5.1                       1_gnu  
binutils_impl_linux-64    2.38                 h2a08ee3_1  
blas                      1.0                         mkl  
bzip2                     1.0.8                h7b6447c_0  
c-ares                    1.19.1               h5eee18b_0  
ca-certificates           2023.11.17           hbcca054_0    conda-forge
cmake                     3.26.4               h96355d8_0  
coverage                  7.2.2           py310h5eee18b_0  
cython                    3.0.0           py310h5eee18b_0  
distro                    1.5.0              pyhd3eb1b0_1  
dpcpp-cpp-rt              2023.1.0         h06a4308_46305  
dpcpp_impl_linux-64       2023.1.0         h6a678d5_46305  
dpcpp_linux-64            2023.1.0         h3a05717_46305  
exceptiongroup            1.0.4           py310h06a4308_0  
expat                     2.5.0                h6a678d5_0  
gcc_impl_linux-64         7.5.0               habd7529_20    conda-forge
gmock                     1.14.0               h06a4308_0  
gtest                     1.14.0               hdb19cb5_0  
gxx_impl_linux-64         7.5.0               hd0bb8aa_20    conda-forge
iniconfig                 1.1.1              pyhd3eb1b0_0  
intel-cmplr-lib-rt        2023.1.0         h06a4308_46305  
intel-cmplr-lic-rt        2023.1.0         h06a4308_46305  
intel-opencl-rt           2023.1.0         h06a4308_46305  
intel-openmp              2023.1.0         hdb19cb5_46306  
kernel-headers_linux-64   2.6.32              he073ed8_16    conda-forge
krb5                      1.20.1               h143b758_1  
ld_impl_linux-64          2.38                 h1181459_1  
libcurl                   8.4.0                h251f7ec_0  
libedit                   3.1.20221030         h5eee18b_0  
libev                     4.33                 h7f8727e_1  
libffi                    3.4.4                h6a678d5_0  
libgcc-devel_linux-64     7.5.0               hda03d7c_20    conda-forge
libgcc-ng                 11.2.0               h1234567_1  
libgomp                   11.2.0               h1234567_1  
libnghttp2                1.57.0               h2d74bed_0  
libssh2                   1.10.0               hdbd6064_2  
libstdcxx-devel_linux-64  7.5.0               hb016644_20    conda-forge
libstdcxx-ng              11.2.0               h1234567_1  
libuuid                   1.41.5               h5eee18b_0  
libuv                     1.44.2               h5eee18b_0  
lz4-c                     1.9.4                h6a678d5_0  
make                      4.2.1                h1bed415_1  
mkl                       2023.1.0         h213fc3f_46344  
mkl-service               2.4.0           py310h5eee18b_1  
mkl_fft                   1.3.8           py310h5eee18b_0  
mkl_random                1.2.4           py310hdb19cb5_0  
ncurses                   6.4                  h6a678d5_0  
ninja                     1.10.2               h06a4308_5  
ninja-base                1.10.2               hd09550d_5  
numpy                     1.26.0          py310h5f9d8c6_0  
numpy-base                1.26.0          py310hb5e798b_0  
openssl                   3.0.12               h7f8727e_0  
packaging                 23.1            py310h06a4308_0  
pip                       23.3.1          py310h06a4308_0  
pluggy                    1.0.0           py310h06a4308_1  
pytest                    7.4.0           py310h06a4308_0  
pytest-cov                4.1.0           py310h06a4308_1  
python                    3.10.13              h955ad1f_0  
readline                  8.2                  h5eee18b_0  
rhash                     1.4.3                hdbd6064_0  
scikit-build              0.15.0          py310h6a678d5_0  
setuptools                68.0.0          py310h06a4308_0  
sqlite                    3.41.2               h5eee18b_0  
sysroot_linux-64          2.12                he073ed8_16    conda-forge
tbb                       2021.8.0             hdb19cb5_0  
tk                        8.6.12               h1ccaba5_0  
toml                      0.10.2             pyhd3eb1b0_0  
tomli                     2.0.1           py310h06a4308_0  
tzdata                    2023c                h04d1e81_0  
wheel                     0.41.2          py310h06a4308_0  
xz                        5.4.2                h5eee18b_0  
zlib                      1.2.13               h5eee18b_0  
zstd                      1.5.5                hc292b87_0
oleksandr-pavlyk commented 9 months ago

@diptorupd Does this issue still need any work?

diptorupd commented 9 months ago

I guess not. I am closing the issue, will reach out if I run into any further build problems.