alex-petrenko / faster-fifo

Faster alternative to Python's multiprocessing.Queue (IPC FIFO queue)
MIT License
179 stars 29 forks source link

Failed building wheel for faster-fifo #49

Closed mxilinas closed 5 months ago

mxilinas commented 5 months ago

Hello, I encountered the following issue while installing with pip. python version: 3.12.3 OS: ArchLinux

Building wheels for collected packages: faster-fifo
  Building wheel for faster-fifo (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for faster-fifo (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [26 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-312
      creating build/lib.linux-x86_64-cpython-312/cpp_faster_fifo
      copying cpp_faster_fifo/__init__.py -> build/lib.linux-x86_64-cpython-312/cpp_faster_fifo
      creating build/lib.linux-x86_64-cpython-312/faster_fifo_reduction
      copying faster_fifo_reduction/__init__.py -> build/lib.linux-x86_64-cpython-312/faster_fifo_reduction
      creating build/lib.linux-x86_64-cpython-312/cpp_faster_fifo/tests
      copying cpp_faster_fifo/tests/__init__.py -> build/lib.linux-x86_64-cpython-312/cpp_faster_fifo/tests
      copying cpp_faster_fifo/tests/comparison_tests.py -> build/lib.linux-x86_64-cpython-312/cpp_faster_fifo/tests
      copying cpp_faster_fifo/tests/test_faster_fifo.py -> build/lib.linux-x86_64-cpython-312/cpp_faster_fifo/tests
      running build_ext
      building 'faster_fifo' extension
      creating build/temp.linux-x86_64-cpython-312
      creating build/temp.linux-x86_64-cpython-312/cpp_faster_fifo
      creating build/temp.linux-x86_64-cpython-312/cpp_faster_fifo/cpp_lib
      gcc -fno-strict-overflow -DNDEBUG -g -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -fPIC -Icpp_faster_fifo/cpp_lib -I/home/michael/gdrl/venv.sf/include -I/usr/include/python3.12 -c cpp_faster_fifo/cpp_lib/faster_fifo.cpp -o build/temp.linux-x86_64-cpython-312/cpp_faster_fifo/cpp_lib/faster_fifo.o -std=c++11
      cpp_faster_fifo/cpp_lib/faster_fifo.cpp: In function ‘void create_queue(void*, size_t, size_t)’:
      cpp_faster_fifo/cpp_lib/faster_fifo.cpp:133:56: error: no matching function for call to ‘operator new(sizetype, void*&)’
        133 |     new(queue_obj_memory) Queue(max_size_bytes, maxsize);
            |                                                        ^
      <built-in>: note: candidate: ‘void* operator new(long unsigned int)’
      <built-in>: note:   candidate expects 1 argument, 2 provided
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for faster-fifo
Failed to build faster-fifo
ERROR: Could not build wheels for faster-fifo, which is required to install pyproject.toml-based projects

I was able to fix it by including \ in faster_fifo.cpp and compiling/installing the package from the tarball.

alex-petrenko commented 5 months ago

👍 thank you for sharing! Let's add !

enchanted-crow commented 5 months ago

Same issue for me. I added #include <new> to the beginning of faster-fifo.cpp and tried to build. But it shows the same error.

Python version: 3.9.19 OS: Fedora 40

$python setup.py build_ext --inplace

Compiling faster_fifo.pyx because it depends on /home/jawad/anaconda3/envs/llarp-2/lib/python3.12/site-packages/Cython/Includes/libcpp/__init__.pxd.
[1/1] Cythonizing faster_fifo.pyx
running build_ext
building 'faster_fifo' extension
creating build/temp.linux-x86_64-cpython-312
creating build/temp.linux-x86_64-cpython-312/cpp_faster_fifo
creating build/temp.linux-x86_64-cpython-312/cpp_faster_fifo/cpp_lib
gcc -pthread -B /home/jawad/anaconda3/envs/llarp-2/compiler_compat -fno-strict-overflow -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /home/jawad/anaconda3/envs/llarp-2/include -fPIC -O2 -isystem /home/jawad/anaconda3/envs/llarp-2/include -fPIC -Icpp_faster_fifo/cpp_lib -I/home/jawad/anaconda3/envs/llarp-2/include/python3.12 -c cpp_faster_fifo/cpp_lib/faster_fifo.cpp -o build/temp.linux-x86_64-cpython-312/cpp_faster_fifo/cpp_lib/faster_fifo.o -std=c++11
cpp_faster_fifo/cpp_lib/faster_fifo.cpp: In function ‘void create_queue(void*, size_t, size_t)’:
cpp_faster_fifo/cpp_lib/faster_fifo.cpp:133:56: error: no matching function for call to ‘operator new(sizetype, void*&)’
  133 |     new(queue_obj_memory) Queue(max_size_bytes, maxsize);
      |                                                        ^
<built-in>: note: candidate: ‘void* operator new(long unsigned int)’
<built-in>: note:   candidate expects 1 argument, 2 provided
error: command '/usr/bin/gcc' failed with exit code 1

I added the include here.

END: Cython Metadata */

#include <new>

#ifndef PY_SSIZE_T_CLEAN
#define PY_SSIZE_T_CLEAN
#endif /* PY_SSIZE_T_CLEAN */
#if defined(CYTHON_LIMITED_API) && 0
  #ifndef Py_LIMITED_API
    #if CYTHON_LIMITED_API+0 > 0x03030000
      #define Py_LIMITED_API CYTHON_LIMITED_API
    #else
      #define Py_LIMITED_API 0x03030000
    #endif
  #endif
#endif

Not sure why it would give the same error after including. Interestingly, after executing the python setup.py command a couple of times, the edited faster-fifo.cpp file reverts to original. Will greatly appreciate any help.

alex-petrenko commented 5 months ago

@mxilinas @enchanted-crow released the new version, should have the missing header https://pypi.org/project/faster-fifo/1.4.7/ Thank you for finding this!