Open floringogianu opened 6 years ago
I was able to compile it and install it manually like this:
cmake -DCMAKE_BUILD_TYPE=Release \
-DBUILD_PYTHON3=ON -DBUILD_JAVA=OFF -DBUILD_LUA=OFF \
-DNUMPY_INCLUDES=~/Tools/miniconda3/envs/torch30/lib/python3.6/site-packages/numpy/core/include/ \
-DPYTHON_EXECUTABLE=~/Tools/miniconda3/envs/torch30/bin/python/ \
-DPYTHON_INCLUDE_DIR=~/Tools/miniconda3/envs/torch30/include/python3.6m/ \
-DPYTHON_LIBRARY=~/Tools/miniconda3/envs/torch30/lib/libpython3.6m.so
make
cp -r bin/python3/pip_package ~/Tools/miniconda3/envs/torch30/lib/python3.6/site-packages/vizdoom
I would leave the issue open though, I'd try to figure out why nor the PyPI method or the setup from sources is working.
Are you sure that the pip you used is the one for python you use?
I never considered checking because that's what conda
virtualenv manages for you. But yes, it seems to be fine:
$ which python
/home/florin/Tools/miniconda3/envs/torch30/bin/python
$ which pip
/home/florin/Tools/miniconda3/envs/torch30/bin/pip
I asked a friend with a different ubuntu version to conda create -n test_env python==3.6; source activate test_env; pip install vizdoom
and he gets the same error.
Problem persists. I set up a fresh 'virtualenv', installed all dependencies via pip3 and pip freeze
does show vizdoom==1.1.4
but importing fails:
(...)
urllib3==1.22
vizdoom==1.1.4
wcwidth==0.1.7
Werkzeug==0.14.1
wrapt==1.10.11
zmq==0.0.0
~/ViZDoom/
(env_3_6) [master…]$: python examples/python/fps.py
Traceback (most recent call last):
File "fps.py", line 16, in <module>
from vizdoom import *
ModuleNotFoundError: No module named 'vizdoom'
@biggzlar Could you give us the exact commands to create the virtualenv?
Thanks for your reply. It’s really just this:
$: virtualenv env_3_6 —python=python3.6
$: . env_3_6/bin/activate
$: pip install numpy vizdoom
No errors or anything.
Fascinating, I am getting the following when I first tried to replicate the error. However when I "uninstall" vizdoom and try to install it again (in the environment), it throws no errors and claims vizdoom is installed correctly.
@mihahauke
Maybe this is related to how sometimes people require to sudo
the pip installation?
virtualenv testenv —python=python3.5
. testenv/bin/activate
pip install numpy vizdoom
[ 99%] Building CXX object src/vizdoom/src/CMakeFiles/vizdoom.dir/viz_message_queue.cpp.o
[ 99%] Building CXX object src/vizdoom/src/CMakeFiles/vizdoom.dir/viz_screen.cpp.o
[ 99%] Building CXX object src/vizdoom/src/CMakeFiles/vizdoom.dir/viz_shared_memory.cpp.o
/tmp/pip-install-r20q55g4/vizdoom/src/vizdoom/src/viz_main.cpp: In function ‘void VIZ_PrintFuncMsg(const char*, const char*)’:
/tmp/pip-install-r20q55g4/vizdoom/src/vizdoom/src/viz_main.cpp:463:23: warning: NULL used in arithmetic [-Wpointer-arith]
while (func[s] != NULL && func[s] != ' ') ++s;
^
/tmp/pip-install-r20q55g4/vizdoom/src/vizdoom/src/viz_main.cpp:465:23: warning: NULL used in arithmetic [-Wpointer-arith]
while (func[e] != NULL && func[e] != '(') ++e;
^
[ 99%] Linking CXX executable ../../../bin/vizdoom
CMakeFiles/vizdoom.dir/tempfiles.cpp.o: In function `FTempFileName::FTempFileName(char const*)':
tempfiles.cpp:(.text+0x7): warning: the use of `tempnam' is dangerous, better use `mkstemp'
[ 99%] Built target vizdoom
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2
Installation failed, you may be missing some dependencies.
Please check https://github.com/mwydmuch/ViZDoom/blob/master/doc/Building.md for details
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-r20q55g4/vizdoom/setup.py", line 119, in <module>
keywords=['vizdoom', 'doom', 'ai', 'deep learning', 'reinforcement learning', 'research']
File "/home/anssk/testenv/testenv36/lib/python3.5/site-packages/setuptools/__init__.py", line 129, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.5/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/home/anssk/testenv/testenv36/lib/python3.5/site-packages/wheel/bdist_wheel.py", line 202, in run
self.run_command('build')
File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/tmp/pip-install-r20q55g4/vizdoom/setup.py", line 82, in run
subprocess.check_call(['make', '-j', str(cpu_cores)])
File "/usr/lib/python3.5/subprocess.py", line 581, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', '-j', '7']' returned non-zero exit status 2
----------------------------------------
Failed building wheel for vizdoom
Running setup.py clean for vizdoom
Failed to build vizdoom
Installing collected packages: numpy, vizdoom
Running setup.py install for vizdoom ... done
Successfully installed numpy-1.14.5 vizdoom-1.1.4
That is weird. Also just tried to run the pip installation with sudo
, again with the same result.
sudo
ing while in environment seems to use the global pip
to install packages (or that's just my setup doing that). I tried to replicate the issue again but now everything seems to install fine, even with pip's --no-cache-dir
option õ.o
What happens if you try to install vizdoom without the environment? Maybe there are some dependencies missing and something suppresses the errors somewhere?
Just tried, the import still fails even when installing without a virtual environment. :/
$: python3 -m pip install vizdoom
Collecting vizdoom
Requirement already satisfied: numpy in /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages (from vizdoom)
Installing collected packages: vizdoom
Successfully installed vizdoom-1.1.4
$: python3 -m pip freeze
(...)
numpy==1.14.5
(...)
vizdoom==1.1.4
Werkzeug==0.14.1
wrapt==1.10.11
$: python3 -c 'import vizdoom'
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'vizdoom'
Hmm that Python path looks peculiar, could be throwing off ViZDoom.
What do the following commands show for you (with and without virtualenv). I think the most perplexing part here is the fact pip
claims ViZDoom is somewhere while Python itself can't find it:
python3 -m site
python3 -m site --user-site
python -c "import numpy; print(numpy.__file__)
Here are the outputs:
$: python3 -m site
sys.path = [
'/Users/biggzlar',
'/Users/biggzlar/Library/Python/2.7/bin',
'/Library/Frameworks/Python.framework/Versions/3.6/lib/python36.zip',
'/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6',
'/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload',
'/Users/biggzlar/Library/Python/3.6/lib/python/site-packages',
'/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages',
]
USER_BASE: '/Users/biggzlar/Library/Python/3.6' (exists)
USER_SITE: '/Users/biggzlar/Library/Python/3.6/lib/python/site-packages' (exists)
ENABLE_USER_SITE: True
$: python3 -m site --user-site
/Users/biggzlar/Library/Python/3.6/lib/python/site-packages
$: python -c "import numpy; print(numpy.__file__)"
/usr/local/lib/python2.7/site-packages/numpy/__init__.pyc
$: python3 -c "import numpy; print(numpy.__file__)"
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/__init__.py
I see what you mean when you say peculiar... Now the same in an environment:
$: python3 -m site
sys.path = [
'/Users/biggzlar',
'/Users/biggzlar/Library/Python/2.7/bin',
'/Users/biggzlar/env3/lib/python36.zip',
'/Users/biggzlar/env3/lib/python3.6',
'/Users/biggzlar/env3/lib/python3.6/lib-dynload',
'/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6',
'/Users/biggzlar/env3/lib/python3.6/site-packages',
]
USER_BASE: '/Users/biggzlar/.local' (exists)
USER_SITE: '/Users/biggzlar/.local/lib/python3.6/site-packages' (exists)
ENABLE_USER_SITE: False
$: python3 -m site --user-site
/Users/biggzlar/.local/lib/python3.6/site-packages
$: python -c "import numpy; print(numpy.__file__)"
/Users/biggzlar/env3/lib/python3.6/site-packages/numpy/__init__.py
Check if you have any vizdoom directories in /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages
. Seems like pip
only cares if the vizdoom-1.1.4.dist-info
directory is present, even if the library itself (vizdoom
) is missing. At this point it sounds like compiling goes well but for some reason the compiled library is not added to Python path. You could try manual compiling like above, should be straight-forward enough.
After uninstalling with pip there are no vizdoom related folders left in both environments. As for the compilation, I am actually now encountering this issue #236
Thank you for your help so far!
I have no experience with Mac stuff, but could this be the issue?
https://stackoverflow.com/questions/16454129/python-h-header-file-missing-on-mac-os-x-10-6
(replace #include <Python.h>
with #include <Python/Python.h>
and add -framework Python
to gcc parameters).
It very well could be, but the solutions don't seem to work. Added the relevant parameter flags to the CMakeLists.txt and ran the build process again as described above - still missing the file.
I then updated the include paths in my current environment, pointing to the headers folder as such:
export C_INCLUDE_PATH=/System/Library/Frameworks/Python.framework/Headers
export CPLUS_INCLUDE_PATH=/System/Library/Frameworks/Python.framework/Headers
No luck.
Things go beyond me at this point :(. From what I gathered around internet the problem seems to the wrong include statement. I would love to debug this myself but closest Mac computer is likely few dozen kilometers away from me currently.
Thanks for your help either way! I'll need to switch machines for now but keep looking into the issue.
The problem persists (Arch Linux, Python 3.7).
It seems to be caused by Python version naming-confusions when building the PIP package: part of the make procedure copies things to bin/python3
and other parts reference bin/python3.7
. This causes silent errors and generates a broken PIP package that nevertheless installs without error.
[100%] Built target pk3
Scanning dependencies of target python_pip_package
[100%] Assembling pip package in ./ViZDoom/bin/python3/pip_package
[100%] Built target python_pip_package
running build_py
package init file 'bin/python3.7/pip_package/__init__.py' not found (or not a regular file)
running egg_info
I've made a quick'n dirty fix by creating a link ln -s python3.7 python3
in ViZDoom/bin
, and then the pip package is actually build successful (and has actual content).
Once the package was installed, I encountered #333 (Cython version-check breaks on 3.7!=3.7.0
). After patching __init__.py
as suggested in the report and rebuilding it worked fine.
Both
pip install vizdoom
andpip install .
from sources terminate without error, however the module is somehow not registered:Any suggestions?