search

openai / doom-py

ViZDoom Python wrapper
MIT License
75 stars 44 forks source link

Not building using Python 3 still #6

Closed t3chman closed 7 years ago

t3chman commented 8 years ago

Related to 320bf0c4b865663e91df55944ae98fed2235808d proposed fix, which built, but had issues with use. I've attempted to build the latest commit and can't get it to build. I've even attempted manually configuring the PYTHON_LIBRARY env variable pointing to the correct python library, but regardless, it's giving the following error:

> pip install .
Processing /home/gabep/downloads/doom-py
Requirement already satisfied (use --upgrade to upgrade): numpy in /env/python/python3/lib/python3.5/site-packages (from doom-py==0.0.13)
Building wheels for collected packages: doom-py
  Running setup.py bdist_wheel for doom-py ... error
  Complete output from command /env/python/python3/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-v1ghs3n6-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /tmp/tmp36e899lapip-wheel- --python-tag cp35:
  running bdist_wheel
  running build
  -- The C compiler identification is GNU 4.9.2
  -- The CXX compiler identification is GNU 4.9.2
  -- Check for working C compiler: /usr/bin/cc
  -- Check for working C compiler: /usr/bin/cc -- works
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working CXX compiler: /usr/bin/c++
  -- Check for working CXX compiler: /usr/bin/c++ -- works
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Boost version: 1.55.0
  -- Found the following Boost libraries:
  --   filesystem
  --   thread
  --   system
  --   date_time
  --   chrono
  --   regex
  --   iostreams
  -- Looking for include file pthread.h
  -- Looking for include file pthread.h - found
  -- Looking for pthread_create
  -- Looking for pthread_create - not found
  -- Looking for pthread_create in pthreads
  -- Looking for pthread_create in pthreads - not found
  -- Looking for pthread_create in pthread
  -- Looking for pthread_create in pthread - found
  -- Found Threads: TRUE
  -- Could NOT find BZip2 (missing:  BZIP2_LIBRARIES BZIP2_INCLUDE_DIR)
  -- Found JPEG: /usr/lib/x86_64-linux-gnu/libjpeg.so
  -- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.8")
  -- Could NOT find GME (missing:  GME_LIBRARIES GME_INCLUDE_DIR)
  -- Using system zlib
  -- Using system jpeg library
  -- Using internal bzip2 library
  -- Using internal gme library
  -- Performing Test HAVE_NO_ARRAY_BOUNDS
  -- Performing Test HAVE_NO_ARRAY_BOUNDS - Success
  -- Performing Test __LIBGME_TEST_VISIBILITY
  -- Performing Test __LIBGME_TEST_VISIBILITY - Success
  -- Looking for strdup
  -- Looking for strdup - found
  -- Looking for strndup
  -- Looking for strndup - found
  -- Looking for sys/types.h
  -- Looking for sys/types.h - found
  -- Looking for stdint.h
  -- Looking for stdint.h - found
  -- Looking for stddef.h
  -- Looking for stddef.h - found
  -- Check size of char
  -- Check size of char - done
  -- Check size of short
  -- Check size of short - done
  -- Check size of int
  -- Check size of int - done
  -- Check size of long
  -- Check size of long - done
  -- /usr/include
  -- /tmp/pip-v1ghs3n6-build/doom_py/src/vizdoom/bzip2
  -- /tmp/pip-v1ghs3n6-build/doom_py/src/vizdoom/lzma/C
  -- Looking for itoa
  -- Looking for itoa - not found
  -- Performing Test DUMB_CAN_USE_SSE
  -- Performing Test DUMB_CAN_USE_SSE - Success
  -- Found PkgConfig: /usr/bin/pkg-config (found version "0.28")
  -- checking for module 'gtk+-2.0'
  --   found gtk+-2.0, version 2.24.25
  -- Found SDL2: /usr/lib/x86_64-linux-gnu/libSDL2main.a;/usr/lib/x86_64-linux-gnu/libSDL2.so;-lpthread
  -- Could NOT find OpenAL (missing:  OPENAL_LIBRARY OPENAL_INCLUDE_DIR)
  -- Could not find FMOD include files
  -- Could NOT find FluidSynth (missing:  FLUIDSYNTH_LIBRARIES FLUIDSYNTH_INCLUDE_DIR)
  -- Selected assembler: /usr/bin/as
  -- Looking for filelength
  -- Looking for filelength - not found
  -- Looking for strupr
  -- Looking for strupr - not found
  -- Looking for stricmp
  -- Looking for stricmp - not found
  -- Looking for strnicmp
  -- Looking for strnicmp - not found
  -- Looking for clock_gettime in rt
  -- Looking for clock_gettime in rt - found
  -- Performing Test HAS_VA_COPY
  -- Performing Test HAS_VA_COPY - Success
  -- Boost version: 1.55.0
  -- Found the following Boost libraries:
  --   thread
  --   system
  --   date_time
  --   chrono
  -- Fluid synth libs: FLUIDSYNTH_LIBRARIES-NOTFOUND
  -- Found PythonInterp: /env/python/python3/bin/python (found suitable version "3.5.1", minimum required is "2.7")
  -- Could NOT find PythonLibs (missing:  PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS) (Required is at least version "3.5")
  -- Using python library: PYTHON_LIBRARY-NOTFOUND
  -- Using python include:
  -- Could NOT find Boost
  -- Could NOT find Boost
  -- Boost library python-py35:
  -- Could NOT find Boost
  -- Boost library python35:
  -- Could NOT find Boost
  -- Boost library python-py3:
  -- Could NOT find Boost
  -- Boost library python3:
  -- Boost version: 1.55.0
  -- Found the following Boost libraries:
  --   python
  -- Found NUMPY: /env/python/python3/lib/python3.5/site-packages/numpy/core/include
  -- Found components for NumPy
  -- NUMPY_ROOT_DIR    = /usr/local
  -- NUMPY_INCLUDES    = /env/python/python3/lib/python3.5/site-packages/numpy/core/include
  -- NUMPY_LIBRARIES   = /env/python/python3/lib/python3.5/site-packages/numpy/core/lib/libnpymath.a
  -- NUMPY_API_VERSION = 1.11.0
  CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
  Please set them or make sure they are set and tested correctly in the CMake files:
  PYTHON_LIBRARY (ADVANCED)
      linked by target "libvizdoom_python" in directory /tmp/pip-v1ghs3n6-build/doom_py

  -- Configuring incomplete, errors occurred!
  See also "/tmp/pip-v1ghs3n6-build/doom_py/CMakeFiles/CMakeOutput.log".
  See also "/tmp/pip-v1ghs3n6-build/doom_py/CMakeFiles/CMakeError.log".

  Could not build doom-py: Command '['cmake', '-DCMAKE_BUILD_TYPE=Release', '-DBUILD_PYTHON=ON', '-DBUILD_JAVA=OFF', '-DPYTHON_EXECUTABLE:FILEPATH=/env/python/python3/bin/python']' returned non-zero exit status 1. (HINT: are you sure cmake is installed? You might also be missing a library. Try running 'apt-get install -y python-numpy cmake zlib1g-dev libjpeg-dev libboost-all-dev gcc libsdl2-dev wget unzip'

  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-v1ghs3n6-build/setup.py", line 86, in <module>
      include_package_data=True,
    File "/env/python/python3/lib/python3.5/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/env/python/python3/lib/python3.5/distutils/dist.py", line 955, in run_commands
      self.run_command(cmd)
    File "/env/python/python3/lib/python3.5/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/env/python/python3/lib/python3.5/site-packages/wheel/bdist_wheel.py", line 179, in run
      self.run_command('build')
    File "/env/python/python3/lib/python3.5/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/env/python/python3/lib/python3.5/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/tmp/pip-v1ghs3n6-build/setup.py", line 61, in run
      build_func()
    File "/tmp/pip-v1ghs3n6-build/setup.py", line 41, in build_linux
      build_common('so')
    File "/tmp/pip-v1ghs3n6-build/setup.py", line 28, in build_common
      subprocess.check_call(['cmake', '-DCMAKE_BUILD_TYPE=Release', '-DBUILD_PYTHON=ON', '-DBUILD_JAVA=OFF', '-DPYTHON_EXECUTABLE:FILEPATH={}'.format(sys.executable)] + cmake_arg_list, cwd='doom_py')
    File "/env/python/python3/lib/python3.5/subprocess.py", line 584, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['cmake', '-DCMAKE_BUILD_TYPE=Release', '-DBUILD_PYTHON=ON', '-DBUILD_JAVA=OFF', '-DPYTHON_EXECUTABLE:FILEPATH=/env/python/python3/bin/python']' returned non-zero exit status 1

  ----------------------------------------
  Failed building wheel for doom-py
  Running setup.py clean for doom-py
Failed to build doom-py
Installing collected packages: doom-py
  Found existing installation: doom-py 0.0.12
    Uninstalling doom-py-0.0.12:
      Successfully uninstalled doom-py-0.0.12
  Running setup.py install for doom-py ... error
    Complete output from command /env/python/python3/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-v1ghs3n6-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-ckiihjof-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    -- Boost version: 1.55.0
    -- Found the following Boost libraries:
    --   filesystem
    --   thread
    --   system
    --   date_time
    --   chrono
    --   regex
    --   iostreams
    -- Could NOT find BZip2 (missing:  BZIP2_LIBRARIES BZIP2_INCLUDE_DIR)
    -- Could NOT find GME (missing:  GME_LIBRARIES GME_INCLUDE_DIR)
    -- Using system zlib
    -- Using system jpeg library
    -- Using internal bzip2 library
    -- Using internal gme library
    -- /usr/include
    -- /tmp/pip-v1ghs3n6-build/doom_py/src/vizdoom/bzip2
    -- /tmp/pip-v1ghs3n6-build/doom_py/src/vizdoom/lzma/C
    -- Could NOT find OpenAL (missing:  OPENAL_LIBRARY OPENAL_INCLUDE_DIR)
    -- Could not find FMOD include files
    -- Could NOT find FluidSynth (missing:  FLUIDSYNTH_LIBRARIES FLUIDSYNTH_INCLUDE_DIR)
    -- Selected assembler: /usr/bin/as
    -- Boost version: 1.55.0
    -- Found the following Boost libraries:
    --   thread
    --   system
    --   date_time
    --   chrono
    -- Fluid synth libs: FLUIDSYNTH_LIBRARIES-NOTFOUND
    -- Could NOT find PythonLibs (missing:  PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS) (Required is at least version "3.5")
    -- Using python library: PYTHON_LIBRARY-NOTFOUND
    -- Using python include:
    -- Could NOT find Boost
    -- Could NOT find Boost
    -- Boost library python-py35:
    -- Could NOT find Boost
    -- Boost library python35:
    -- Could NOT find Boost
    -- Boost library python-py3:
    -- Could NOT find Boost
    -- Boost library python3:
    -- Boost version: 1.55.0
    -- Found the following Boost libraries:
    --   python
    -- Found components for NumPy
    -- NUMPY_ROOT_DIR    = /usr/local
    -- NUMPY_INCLUDES    = /env/python/python3/lib/python3.5/site-packages/numpy/core/include
    -- NUMPY_LIBRARIES   = /env/python/python3/lib/python3.5/site-packages/numpy/core/lib/libnpymath.a
    -- NUMPY_API_VERSION = 1.11.0
    CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
    Please set them or make sure they are set and tested correctly in the CMake files:
    PYTHON_LIBRARY (ADVANCED)
        linked by target "libvizdoom_python" in directory /tmp/pip-v1ghs3n6-build/doom_py

    -- Configuring incomplete, errors occurred!
    See also "/tmp/pip-v1ghs3n6-build/doom_py/CMakeFiles/CMakeOutput.log".
    See also "/tmp/pip-v1ghs3n6-build/doom_py/CMakeFiles/CMakeError.log".

    Could not build doom-py: Command '['cmake', '-DCMAKE_BUILD_TYPE=Release', '-DBUILD_PYTHON=ON', '-DBUILD_JAVA=OFF', '-DPYTHON_EXECUTABLE:FILEPATH=/env/python/python3/bin/python']' returned non-zero exit status 1. (HINT: are you sure cmake is installed? You might also be missing a library. Try running 'apt-get install -y python-numpy cmake zlib1g-dev libjpeg-dev libboost-all-dev gcc libsdl2-dev wget unzip'

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-v1ghs3n6-build/setup.py", line 86, in <module>
        include_package_data=True,
      File "/env/python/python3/lib/python3.5/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/env/python/python3/lib/python3.5/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/env/python/python3/lib/python3.5/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/env/python/python3/lib/python3.5/site-packages/setuptools-22.0.5-py3.5.egg/setuptools/command/install.py", line 61, in run
      File "/env/python/python3/lib/python3.5/distutils/command/install.py", line 539, in run
        self.run_command('build')
      File "/env/python/python3/lib/python3.5/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/env/python/python3/lib/python3.5/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/tmp/pip-v1ghs3n6-build/setup.py", line 61, in run
        build_func()
      File "/tmp/pip-v1ghs3n6-build/setup.py", line 41, in build_linux
        build_common('so')
      File "/tmp/pip-v1ghs3n6-build/setup.py", line 28, in build_common
        subprocess.check_call(['cmake', '-DCMAKE_BUILD_TYPE=Release', '-DBUILD_PYTHON=ON', '-DBUILD_JAVA=OFF', '-DPYTHON_EXECUTABLE:FILEPATH={}'.format(sys.executable)] + cmake_arg_list, cwd='doom_py')
      File "/env/python/python3/lib/python3.5/subprocess.py", line 584, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['cmake', '-DCMAKE_BUILD_TYPE=Release', '-DBUILD_PYTHON=ON', '-DBUILD_JAVA=OFF', '-DPYTHON_EXECUTABLE:FILEPATH=/env/python/python3/bin/python']' returned non-zero exit status 1

    ----------------------------------------
  Rolling back uninstall of doom-py
Command "/env/python/python3/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-v1ghs3n6-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-ckiihjof-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-v1ghs3n6-build/
t3chman commented 8 years ago

@jietang hey, not sure if you saw this, but thought I should tag you just in case. Decided opening an issue was probably more effective than sorting through chat logs on gitter :)

jietang commented 8 years ago

Good idea, let's try to debug this.

Agree with you that it's not finding the Python 3.5 library. What version of CMake are you using (cmake --version)?

Can you try running in a python3 shell

import sysconfig sysconfig.get_config_var('LIBPL') '...' sysconfig.get_config_var('LIBDIR') '...' and report the results?

Can you also report the value of env | grep PATH

Looking later in the log, it's not finding boost-python3. If you're on OSX you can install it with

brew install boost-python --with-python3

t3chman commented 8 years ago

I'm running Debian 8 x64. CMake 3.0.2. Python 3.5.1. I'm using anaconda3 - for full disclosure, but it shouldn't be affecting it, unless doom-py or dependencies have hard coded locations, since I have python installed into various directories and activate/deactivate the virtual env's. I'll also list the other env variables I export when activating the python3 env, but they shouldn't be required once we get the issue figured out.

PATH=/env/python/python3/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
PYTHON_LIBRARIES=/env/python/python3/lib/python3.5
PYTHON_INCLUDE_DIRS=/env/python/python3/include/python3.5m

I'll see if I can build boost-python3.

Other results:

> python
Python 3.5.1 |Continuum Analytics, Inc.| (default, Dec  7 2015, 11:16:01) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_config_var('LIBPL')
'/env/python/python3/lib/python3.5/config-3.5m'
>>> sysconfig.get_config_var('LIBDIR')
'/env/python/python3/lib'
jietang commented 8 years ago

I found a reference http://askubuntu.com/a/515485 that suggested that cmake 3.0 only looks up to python3.4.

Can you try building after applying the following patch to doom_py https://gist.github.com/jietang/41723d0d7905e22e5b33a3c7ad455e08

t3chman commented 8 years ago

Good find! Yeah, I'll see if I can make some time to test that this afternoon. I'll be heading out of town for the rest of the week though, so if I don't get back to you Today, I'll get back to you this weekend.

tlbtlbtlb commented 7 years ago

Stale

AlexKaravaev commented 6 years ago

For me installing ZDoom deps (https://zdoom.org/wiki/Compile_ZDoom_on_Linux) seems to have fixed the issue.