search

Daniil-Osokin / lightweight-human-pose-estimation-3d-demo.pytorch

Real-time 3D multi-person pose estimation demo in PyTorch. OpenVINO backend can be used for fast inference on CPU.
Apache License 2.0
653 stars 137 forks source link

Setup issue / NumPy headers not found #71

Closed ryo-matsuzaka closed 3 years ago

ryo-matsuzaka commented 3 years ago

When I run python setup.py build_ext, I got this issue.

My environment: WSL2: Ubuntu18.04 OpenCV: Install according to this cmake version 3.10.2

ryo-matsuzaka@DESKTOP-5CM12VV:~/lightweight-human-pose-estimation-3d-demo.pytorch$ python setup.py build_ext running build_ext -- Found PythonInterp: /home/ryo-matsuzaka/.pyenv/versions/3.5.4/bin/python (found suitable version "3.5.4", minimum required is "3.5") CMake Error at CMakeLists.txt:13 (message): NumPy headers not found

-- Configuring incomplete, errors occurred! See also "/home/ryo-matsuzaka/lightweight-human-pose-estimation-3d-demo.pytorch/pose_extractor/build/tmp/CMakeFiles/CMakeOutput.log". Traceback (most recent call last): File "setup.py", line 72, in cmdclass={'build_ext': CMakeBuild}) File "/home/ryo-matsuzaka/.pyenv/versions/3.5.4/lib/python3.5/distutils/core.py", line 148, in setup dist.run_commands() File "/home/ryo-matsuzaka/.pyenv/versions/3.5.4/lib/python3.5/distutils/dist.py", line 955, in run_commands self.run_command(cmd) File "/home/ryo-matsuzaka/.pyenv/versions/3.5.4/lib/python3.5/distutils/dist.py", line 974, in run_command cmd_obj.run() File "/home/ryo-matsuzaka/.pyenv/versions/3.5.4/lib/python3.5/site-packages/setuptools/command/build_ext.py", line 75, in run _build_ext.run(self) File "/home/ryo-matsuzaka/.pyenv/versions/3.5.4/lib/python3.5/distutils/command/build_ext.py", line 339, in run self.build_extensions() File "setup.py", line 63, in build_extensions subprocess.check_call(['cmake', ext.cmake_lists_dir] + cmake_args, cwd=tmp_dir) File "/home/ryo-matsuzaka/.pyenv/versions/3.5.4/lib/python3.5/subprocess.py", line 271, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['cmake', '/home/ryo-matsuzaka/lightweight-human-pose-estimation-3d-demo.pytorch/pose_extractor', '-DCMAKE_BUILD_TYPE=Release', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY_RELEASE=/home/ryo-matsuzaka/lightweight-human-pose-estimation-3d-demo.pytorch/pose_extractor/build', '-DCMAKE_ARCHIVE_OUTPUT_DIRECTORY_RELEASE=/home/ryo-matsuzaka/lightweight-human-pose-estimation-3d-demo.pytorch/pose_extractor/build/tmp', '-DPYTHON_EXECUTABLE=/home/ryo-matsuzaka/.pyenv/versions/3.5.4/bin/python']' returned non-zero exit status 1

ryo-matsuzaka commented 3 years ago

According to this, I run python -m pip install numpy. Then I got another error.

ryo-matsuzaka@DESKTOP-5CM12VV:~/lightweight-human-pose-estimation-3d-demo.pytorch$ python setup.py build_ext
running build_ext
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ryo-matsuzaka/lightweight-human-pose-estimation-3d-demo.pytorch/pose_extractor/build/tmp
[ 20%] Building CXX object CMakeFiles/pose_extractor.dir/wrapper.cpp.o
/home/ryo-matsuzaka/lightweight-human-pose-estimation-3d-demo.pytorch/pose_extractor/wrapper.cpp:2:10: fatal error: Python.h: No such file or directory
 #include <Python.h>
          ^~~~~~~~~~
compilation terminated.
CMakeFiles/pose_extractor.dir/build.make:62: recipe for target 'CMakeFiles/pose_extractor.dir/wrapper.cpp.o' failed
make[2]: *** [CMakeFiles/pose_extractor.dir/wrapper.cpp.o] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/pose_extractor.dir/all' failed
make[1]: *** [CMakeFiles/pose_extractor.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2
Traceback (most recent call last):
  File "setup.py", line 72, in <module>
    cmdclass={'build_ext': CMakeBuild})
  File "/home/ryo-matsuzaka/.pyenv/versions/3.5.4/lib/python3.5/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/home/ryo-matsuzaka/.pyenv/versions/3.5.4/lib/python3.5/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/home/ryo-matsuzaka/.pyenv/versions/3.5.4/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/ryo-matsuzaka/.pyenv/versions/3.5.4/lib/python3.5/site-packages/setuptools/command/build_ext.py", line 75, in run
    _build_ext.run(self)
  File "/home/ryo-matsuzaka/.pyenv/versions/3.5.4/lib/python3.5/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "setup.py", line 64, in build_extensions
    subprocess.check_call(['cmake', '--build', '.', '--config', cfg], cwd=tmp_dir)
  File "/home/ryo-matsuzaka/.pyenv/versions/3.5.4/lib/python3.5/subprocess.py", line 271, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release']' returned non-zero exit status 2
Daniil-Osokin commented 3 years ago

Hi! Try to install dev package for Python: sudo apt-get install python3-dev.

Daniil-Osokin commented 3 years ago

Hope, it helped.

ryo-matsuzaka commented 3 years ago

Thank you very much. I am late to response your message due to holiday. I have already installed python3-dev when the issue happened. So I keep trying to tackle this issue.

ryo-matsuzaka commented 3 years ago

@Daniil-Osokin Could you reopen this issue?

ryo-matsuzaka commented 3 years ago

https://stackoverflow.com/questions/11041299/python-h-no-such-file-or-directory

ryo-matsuzaka commented 3 years ago

This may be related. I will try it.

Daniil-Osokin commented 3 years ago

As I get you are trying to run it in Windows through WSL? If so, I believe it is better to look for WSL-specific paths to include Python.h. You can try to run find / -name "Python.h"(may be long, but will find you correct path to include). In any case you can run the demo as is (without building anything), just run it.

ryo-matsuzaka commented 3 years ago

As I get you are trying to run it in Windows through WSL?

Yes, I use WSL2.

In any case you can run the demo as is (without building anything), just run it.

Thank you very much! I could run demo.py!

ryo-matsuzaka commented 3 years ago

But I cannot use fast inference as I get the same error like this issue.

ryo-matsuzaka commented 3 years ago

As I get you are trying to run it in Windows through WSL? If so, I believe it is better to look for WSL-specific paths to include Python.h. You can try to run find / -name "Python.h"(may be long, but will find you correct path to include).

I run find / -name "Python.h" And I find the path to Python.h

Then, I add path export CPATH=:/home/ryo-matsuzaka/.pyenv/versions/3.6.13/include/python3.6m/

I can solve one issue but I got another one.

I got this new error message.

running build_ext -- Configuring done -- Generating done -- Build files have been written to: /home/ryo-matsuzaka/lightweight-human-pose-estimation-3d-demo.pytorch/pose_extractor/build/tmp [ 20%] Building CXX object CMakeFiles/pose_extractor.dir/wrapper.cpp.o [ 40%] Building CXX object CMakeFiles/pose_extractor.dir/src/extract_poses.cpp.o [ 60%] Building CXX object CMakeFiles/pose_extractor.dir/src/human_pose.cpp.o [ 80%] Building CXX object CMakeFiles/pose_extractor.dir/src/peak.cpp.o make[2]: No rule to make target '/home/ryo-matsuzaka/.pyenv/versions/3.8.8/lib/libpython3.8.a', needed by '../pose_extractor.so'. Stop. CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/pose_extractor.dir/all' failed make[1]: [CMakeFiles/pose_extractor.dir/all] Error 2 Makefile:83: recipe for target 'all' failed make: * [all] Error 2 Traceback (most recent call last): File "setup.py", line 72, in cmdclass={'build_ext': CMakeBuild}) File "/home/ryo-matsuzaka/.pyenv/versions/3.6.13/lib/python3.6/site-packages/setuptools/init.py", line 143, in setup return distutils.core.setup(attrs) File "/home/ryo-matsuzaka/.pyenv/versions/3.6.13/lib/python3.6/distutils/core.py", line 148, in setup dist.run_commands() File "/home/ryo-matsuzaka/.pyenv/versions/3.6.13/lib/python3.6/distutils/dist.py", line 955, in run_commands self.run_command(cmd) File "/home/ryo-matsuzaka/.pyenv/versions/3.6.13/lib/python3.6/distutils/dist.py", line 974, in run_command cmd_obj.run() File "/home/ryo-matsuzaka/.pyenv/versions/3.6.13/lib/python3.6/site-packages/setuptools/command/build_ext.py", line 78, in run _build_ext.run(self) File "/home/ryo-matsuzaka/.pyenv/versions/3.6.13/lib/python3.6/distutils/command/build_ext.py", line 339, in run self.build_extensions() File "setup.py", line 64, in build_extensions subprocess.check_call(['cmake', '--build', '.', '--config', cfg], cwd=tmp_dir) File "/home/ryo-matsuzaka/.pyenv/versions/3.6.13/lib/python3.6/subprocess.py", line 311, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release']' returned non-zero exit status 2.

Daniil-Osokin commented 3 years ago

I am not sure if this is a proper path. Mine is here: /usr/include/python3.6m/, what is the output of whereis python3?

ryo-matsuzaka commented 3 years ago

python3: /usr/bin/python3 /usr/bin/python3.6 /usr/bin/python3.6m-config /usr/bin/python3.6-config /usr/bin/python3.6m /usr/lib/python3.8 /usr/lib/python3.7 /usr/lib/python3 /usr/lib/python3.6 /etc/python3 /etc/python3.6 /usr/local/lib/python3.6 /usr/include/python3.6 /usr/include/python3.6m /usr/share/python3 /home/ryo-matsuzaka/.pyenv/shims/python3 /home/ryo-matsuzaka/.pyenv/shims/python3.6 /home/ryo-matsuzaka/.pyenv/shims/python3.6m-config /home/ryo-matsuzaka/.pyenv/shims/python3.6-config /home/ryo-matsuzaka/.pyenv/shims/python3.6m /mnt/c/Users/ryo_matsuzaka/AppData/Local/Microsoft/WindowsApps/python3.exe /mnt/c/Users/ryo_matsuzaka/anaconda3/python3.dll /mnt/c/Users/ryo_matsuzaka/anaconda3/python37.dll /mnt/c/Users/ryo_matsuzaka/anaconda3/python37.pdb /usr/share/man/man1/python3.1.gz

ryo-matsuzaka commented 3 years ago

Thank you for quick response. Sorry for taking your time. There is /usr/include/python3.6m/

So I add this to the path, then I run python setup.py build_ext. I got following...

running build_ext -- Configuring done -- Generating done -- Build files have been written to: /home/ryo-matsuzaka/lightweight-human-pose-estimation-3d-demo.pytorch/pose_extractor/build/tmp make[2]: No rule to make target '/home/ryo-matsuzaka/.pyenv/versions/3.8.8/lib/libpython3.8.a', needed by '../pose_extractor.so'. Stop. CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/pose_extractor.dir/all' failed make[1]: [CMakeFiles/pose_extractor.dir/all] Error 2 Makefile:83: recipe for target 'all' failed make: * [all] Error 2 Traceback (most recent call last): File "setup.py", line 72, in cmdclass={'build_ext': CMakeBuild}) File "/home/ryo-matsuzaka/.pyenv/versions/3.6.13/lib/python3.6/site-packages/setuptools/init.py", line 143, in setup return distutils.core.setup(attrs) File "/home/ryo-matsuzaka/.pyenv/versions/3.6.13/lib/python3.6/distutils/core.py", line 148, in setup dist.run_commands() File "/home/ryo-matsuzaka/.pyenv/versions/3.6.13/lib/python3.6/distutils/dist.py", line 955, in run_commands self.run_command(cmd) File "/home/ryo-matsuzaka/.pyenv/versions/3.6.13/lib/python3.6/distutils/dist.py", line 974, in run_command cmd_obj.run() File "/home/ryo-matsuzaka/.pyenv/versions/3.6.13/lib/python3.6/site-packages/setuptools/command/build_ext.py", line 78, in run _build_ext.run(self) File "/home/ryo-matsuzaka/.pyenv/versions/3.6.13/lib/python3.6/distutils/command/build_ext.py", line 339, in run self.build_extensions() File "setup.py", line 64, in build_extensions subprocess.check_call(['cmake', '--build', '.', '--config', cfg], cwd=tmp_dir) File "/home/ryo-matsuzaka/.pyenv/versions/3.6.13/lib/python3.6/subprocess.py", line 311, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release']' returned non-zero exit status 2.

And if I changed the path to the before I cannot reproduce the result. I got the same message above.

ryo-matsuzaka commented 3 years ago

Probably my pyenv environment is not good.

Daniil-Osokin commented 3 years ago

Probably, Python's versions have to be the same. Try to completely remove /home/ryo-matsuzaka/lightweight-human-pose-estimation-3d-demo.pytorch/pose_extractor/build/ and rebuild.

ryo-matsuzaka commented 3 years ago

Thank you very much for gentle reply. I removed build directory and rebuild then I got following message.

ryo-matsuzaka@DESKTOP-5CM12VV:~/lightweight-human-pose-estimation-3d-demo.pytorch$ python setup.py build_ext running build_ext -- The C compiler identification is GNU 7.5.0 -- The CXX compiler identification is GNU 7.5.0 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Found PythonInterp: /home/ryo-matsuzaka/.pyenv/versions/3.8.8/bin/python (found suitable version "3.8.8", minimum required is "3.5") -- Found PythonLibs: /home/ryo-matsuzaka/.pyenv/versions/3.8.8/lib/libpython3.8.a (found suitable version "3.8.8", minimum required is "3.5") -- Found OpenCV: /usr/local (found suitable version "4.5.2", minimum required is "4") -- Configuring done -- Generating done -- Build files have been written to: /home/ryo-matsuzaka/lightweight-human-pose-estimation-3d-demo.pytorch/pose_extractor/build/tmp Scanning dependencies of target pose_extractor [ 20%] Building CXX object CMakeFiles/pose_extractor.dir/wrapper.cpp.o [ 40%] Building CXX object CMakeFiles/pose_extractor.dir/src/extract_poses.cpp.o [ 60%] Building CXX object CMakeFiles/pose_extractor.dir/src/human_pose.cpp.o [ 80%] Building CXX object CMakeFiles/pose_extractor.dir/src/peak.cpp.o [100%] Linking CXX shared library ../pose_extractor.so /usr/bin/ld: /home/ryo-matsuzaka/.pyenv/versions/3.8.8/lib/libpython3.8.a(_warnings.o): relocation R_X86_64_PC32 against symbol `_PyRuntime' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: final link failed: Bad value collect2: error: ld returned 1 exit status CMakeFiles/pose_extractor.dir/build.make:227: recipe for target '../pose_extractor.so' failed make[2]: [../pose_extractor.so] Error 1 CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/pose_extractor.dir/all' failed make[1]: [CMakeFiles/pose_extractor.dir/all] Error 2 Makefile:83: recipe for target 'all' failed make: * [all] Error 2 Traceback (most recent call last): File "setup.py", line 67, in setup(name=PACKAGE_NAME, File "/home/ryo-matsuzaka/.pyenv/versions/3.8.8/lib/python3.8/site-packages/setuptools/init.py", line 165, in setup return distutils.core.setup(attrs) File "/home/ryo-matsuzaka/.pyenv/versions/3.8.8/lib/python3.8/distutils/core.py", line 148, in setup dist.run_commands() File "/home/ryo-matsuzaka/.pyenv/versions/3.8.8/lib/python3.8/distutils/dist.py", line 966, in run_commands self.run_command(cmd) File "/home/ryo-matsuzaka/.pyenv/versions/3.8.8/lib/python3.8/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/home/ryo-matsuzaka/.pyenv/versions/3.8.8/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 87, in run _build_ext.run(self) File "/home/ryo-matsuzaka/.pyenv/versions/3.8.8/lib/python3.8/distutils/command/build_ext.py", line 340, in run self.build_extensions() File "setup.py", line 64, in build_extensions subprocess.check_call(['cmake', '--build', '.', '--config', cfg], cwd=tmp_dir) File "/home/ryo-matsuzaka/.pyenv/versions/3.8.8/lib/python3.8/subprocess.py", line 364, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release']' returned non-zero exit status 2.

Daniil-Osokin commented 3 years ago

I believe you are still having issues with different Python versions. On Ubuntu libpose_extractor.so is linked against shared python library (/usr/lib/x86_64-linux-gnu/libpython3.6m.so in my case). I also see, that you probably have python3.6 as a system one. So python3-dev package was installed also for 3.6 version. And error log shows, that your environment is using 3.8.8 version. So try to make the proper environment (virtualenv -p /usr/bin/python3.6 venv_pose3d) and use it with this repository.

ryo-matsuzaka commented 3 years ago

Thank you very much. I will try.

ryo-matsuzaka commented 3 years ago

Thank you very much! I delete .pyenv directory and create environment using virtualenv as you mentioned above. I could successfully build.