Closed coderkhaleesi closed 3 years ago
urw7rs
commented
4 years ago I think cmake couldn't find json-c. Can you show me what commands you ran? And also make sure you're using ubuntu 18.04.
If you want a quick fix, you can use docker. After installing docker, run $ docker build -t spiralpp . && docker run --name spiralpp -it -p 8888:8888 spiralpp /bin/bash
0xisk
commented
3 years ago Hi @urw7rs, I faced with this error after running this docker command in Ubuntu 18.04:
docker build -t spiralpp . && docker run --name spiralpp -it -p 8888:8888 spiralpp /bin/bashStep 21/39 : RUN pip install -e .
---> Running in 6e0df36dd695
Obtaining file:///src/spiralpp/spiral-envs
Requirement already satisfied: six in /root/miniconda3/envs/spiralpp/lib/python3.7/site-packages (from spiral==1.0) (1.16.0)
Requirement already satisfied: scipy in /root/miniconda3/envs/spiralpp/lib/python3.7/site-packages (from spiral==1.0) (1.7.0)
Requirement already satisfied: numpy in /root/miniconda3/envs/spiralpp/lib/python3.7/site-packages (from spiral==1.0) (1.20.2)
Collecting gym
Downloading gym-0.18.3.tar.gz (1.6 MB)
Collecting pyglet<=1.5.15,>=1.4.0
Downloading pyglet-1.5.15-py3-none-any.whl (1.1 MB)
Collecting Pillow<=8.2.0
Downloading Pillow-8.2.0-cp37-cp37m-manylinux1_x86_64.whl (3.0 MB)
Collecting cloudpickle<1.7.0,>=1.2.0
Downloading cloudpickle-1.6.0-py3-none-any.whl (23 kB)
Building wheels for collected packages: gym
Building wheel for gym (setup.py): started
Building wheel for gym (setup.py): finished with status 'done'
Created wheel for gym: filename=gym-0.18.3-py3-none-any.whl size=1657515 sha256=d2a6b44432cdd1951680213861f4217a5aab4f879c096a7c23bd935ad03d73f3
Stored in directory: /root/.cache/pip/wheels/1a/ec/6d/705d53925f481ab70fd48ec7728558745eeae14dfda3b49c99
Successfully built gym
Installing collected packages: pyglet, Pillow, cloudpickle, gym, spiral
Running setup.py develop for spiral
ERROR: Command errored out with exit status 1:
command: /root/miniconda3/envs/spiralpp/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/src/spiralpp/spiral-envs/setup.py'"'"'; __file__='"'"'/src/spiralpp/spiral-envs/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
cwd: /src/spiralpp/spiral-envs/
Complete output (64 lines):
running develop
running egg_info
creating spiral.egg-info
writing spiral.egg-info/PKG-INFO
writing dependency_links to spiral.egg-info/dependency_links.txt
writing requirements to spiral.egg-info/requires.txt
writing top-level names to spiral.egg-info/top_level.txt
writing manifest file 'spiral.egg-info/SOURCES.txt'
reading manifest file 'spiral.egg-info/SOURCES.txt'
writing manifest file 'spiral.egg-info/SOURCES.txt'
running build_ext
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at /root/miniconda3/envs/spiralpp/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:218 (message):
Could NOT find Python3 (missing: Python3_NumPy_INCLUDE_DIRS NumPy) (found
version "3.8.5")
Call Stack (most recent call first):
/root/miniconda3/envs/spiralpp/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:582 (_FPHSA_FAILURE_MESSAGE)
/root/miniconda3/envs/spiralpp/share/cmake-3.19/Modules/FindPython/Support.cmake:3148 (find_package_handle_standard_args)
/root/miniconda3/envs/spiralpp/share/cmake-3.19/Modules/FindPython3.cmake:398 (include)
CMakeLists.txt:20 (find_package)
-- Configuring incomplete, errors occurred!
See also "/src/spiralpp/spiral-envs/build/temp.linux-x86_64-3.7/CMakeFiles/CMakeOutput.log".
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/src/spiralpp/spiral-envs/setup.py", line 91, in <module>
cmdclass={"build_ext": CMakeBuildExt},
File "/root/miniconda3/envs/spiralpp/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/root/miniconda3/envs/spiralpp/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/root/miniconda3/envs/spiralpp/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/root/miniconda3/envs/spiralpp/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/root/miniconda3/envs/spiralpp/lib/python3.7/site-packages/setuptools/command/develop.py", line 34, in run
self.install_for_development()
File "/root/miniconda3/envs/spiralpp/lib/python3.7/site-packages/setuptools/command/develop.py", line 136, in install_for_development
self.run_command('build_ext')
File "/root/miniconda3/envs/spiralpp/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/root/miniconda3/envs/spiralpp/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/src/spiralpp/spiral-envs/setup.py", line 44, in run
self.build_extension(ext)
File "/src/spiralpp/spiral-envs/setup.py", line 47, in build_extension
self.configure_cmake(ext)
File "/src/spiralpp/spiral-envs/setup.py", line 70, in configure_cmake
subprocess.check_call(configure_cmd, cwd=self.build_temp)
File "/root/miniconda3/envs/spiralpp/lib/python3.7/subprocess.py", line 363, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '/src/spiralpp/spiral-envs', '-DCMAKE_BUILD_TYPE=Release', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY_RELEASE=/src/spiralpp/spiral-envs/spiral/envs', '-DCMAKE_ARCHIVE_OUTPUT_DIRECTORY_RELEASE=build/temp.linux-x86_64-3.7', '-DPYTHON_EXECUTABLE:FILEPATH=/root/miniconda3/envs/spiralpp/bin/python']' returned non-zero exit status 1.
----------------------------------------
ERROR: Command errored out with exit status 1: /root/miniconda3/envs/spiralpp/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/src/spiralpp/spiral-envs/setup.py'"'"'; __file__='"'"'/src/spiralpp/spiral-envs/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.
The command '/bin/bash -c pip install -e .' returned a non-zero code: 1Could you please help me to solve this.
urw7rs
commented
3 years ago I tried building the image and I got the same error. I'll try to fix it.
urw7rs
commented
3 years ago @coderkhaleesi, @iskanderandrews installing cmake 3.14 seems to solve the issue. I could build the docker image without errors.
0xisk
commented
3 years ago @urw7rs Thanks so much for your helping on this.
I rechecked again and yea it pass the posted error, but got another new error:
[4109/4319] Building CXX object test_api/CMakeFiles/test_api.dir/dataloader.cpp.o
FAILED: test_api/CMakeFiles/test_api.dir/dataloader.cpp.o
/usr/bin/c++ -DHAVE_MALLOC_USABLE_SIZE=1 -DHAVE_MMAP=1 -DHAVE_SHM_OPEN=1 -DHAVE_SHM_UNLINK=1 -DIDEEP_USE_MKL -DMINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -DONNXIFI_ENABLE_EXT=1 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DTH_BLAS_MKL -DUSE_EXTERNAL_MZCRC -D_FILE_OFFSET_BITS=64 -Iaten/src -I../aten/src -I. -I../ -I../cmake/../third_party/benchmark/include -Icaffe2/contrib/aten -I../third_party/onnx -Ithird_party/onnx -I../third_party/foxi -Ithird_party/foxi -I../caffe2/../torch/../aten/src -Icaffe2/../aten/src -Icaffe2/../aten/src/ATen -I../caffe2/../torch/csrc/api -I../caffe2/../torch/csrc/api/include -I../c10/.. -Ithird_party/ideep/mkl-dnn/include -I../third_party/ideep/mkl-dnn/src/../include -isystem third_party/gloo -isystem ../cmake/../third_party/gloo -isystem ../cmake/../third_party/googletest/googlemock/include -isystem ../cmake/../third_party/googletest/googletest/include -isystem ../third_party/protobuf/src -isystem /root/miniconda3/envs/spiralpp/include -isystem ../third_party/gemmlowp -isystem ../third_party/neon2sse -isystem ../third_party/XNNPACK/include -isystem ../third_party -isystem ../cmake/../third_party/eigen -isystem /root/miniconda3/envs/spiralpp/include/python3.7m -isystem /root/miniconda3/envs/spiralpp/lib/python3.7/site-packages/numpy/core/include -isystem ../cmake/../third_party/pybind11/include -isystem ../third_party/ideep/mkl-dnn/include -isystem ../third_party/ideep/include -isystem ../third_party/googletest/googletest/include -isystem ../third_party/googletest/googletest -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_VULKAN_WRAPPER -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow -DHAVE_AVX_CPU_DEFINITION -DHAVE_AVX2_CPU_DEFINITION -O3 -DNDEBUG -DNDEBUG -fPIE -DCAFFE2_USE_GLOO -DHAVE_GCC_GET_CPUID -DUSE_AVX -DUSE_AVX2 -DTH_HAVE_THREAD -Wno-maybe-uninitialized -Wno-unused-but-set-parameter -std=c++14 -pthread -std=gnu++14 -MD -MT test_api/CMakeFiles/test_api.dir/dataloader.cpp.o -MF test_api/CMakeFiles/test_api.dir/dataloader.cpp.o.d -o test_api/CMakeFiles/test_api.dir/dataloader.cpp.o -c ../test/cpp/api/dataloader.cpp
c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
[4110/4319] Building CXX object test_api/CMakeFiles/test_api.dir/integration.cpp.o
[4111/4319] Building CXX object test_api/CMakeFiles/test_api.dir/functional.cpp.o
ninja: build stopped: subcommand failed.
Building wheel torch-1.6.0a0+b31f58d
-- Building version 1.6.0a0+b31f58d
cmake -GNinja -DBUILD_PYTHON=True -DBUILD_TEST=True -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/src/pytorch/torch -DCMAKE_PREFIX_PATH=/root/miniconda3/envs/spiralpp -DNUMPY_INCLUDE_DIR=/root/miniconda3/envs/spiralpp/lib/python3.7/site-packages/numpy/core/include -DPYTHON_EXECUTABLE=/root/miniconda3/envs/spiralpp/bin/python -DPYTHON_INCLUDE_DIR=/root/miniconda3/envs/spiralpp/include/python3.7m -DPYTHON_LIBRARY=/root/miniconda3/envs/spiralpp/lib/libpython3.7m.so.1.0 -DTORCH_BUILD_VERSION=1.6.0a0+b31f58d -DUSE_NUMPY=True /src/pytorch
cmake --build . --target install --config Release -- -j 3
Traceback (most recent call last):
File "setup.py", line 732, in <module>
build_deps()
File "setup.py", line 316, in build_deps
cmake=cmake)
File "/src/pytorch/tools/build_pytorch_libs.py", line 62, in build_caffe2
cmake.build(my_env)
File "/src/pytorch/tools/setup_helpers/cmake.py", line 345, in build
self.run(build_args, my_env)
File "/src/pytorch/tools/setup_helpers/cmake.py", line 141, in run
check_call(command, cwd=self.build_dir, env=env)
File "/root/miniconda3/envs/spiralpp/lib/python3.7/subprocess.py", line 363, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '3']' returned non-zero exit status 1.
The command '/bin/bash -c python setup.py install' returned a non-zero code: 1
Alright @urw7rs so I retried the same steps again in a different VM (Arch Linux), and the docker image built very well without any issues. And just now trying to download the Dataset manually celebahq in order to build the model.
So I think you can close this issue. Thanks so much for your helping.
urw7rs
commented
3 years ago I tried to install pytorch using pip but grpc caused errors when building torchbeast. For now, Installing pytorch from source seems to be the only way. I'm glad I could help. I"ll close this issue and the badzipfile issue too.
I am not sure why I am getting this error. I am not sure why the variables aren't getting set.
-- Configuring done CMake Error: The following variables are used in this project, but they are set to NOTFOUND. Please set them or make sure they are set and tested correctly in the CMake files: JSONC_LIBRARY linked by target "pylibmypaint" in directory /students/u7043565/play/spiralpp/spiral-envs/spiral X11 linked by target "OGLES2HelloAPI" in directory /students/u7043565/play/spiralpp/spiral-envs/third_party/swiftshader
ERROR: Command errored out with exit status 1: /students/u7043565/miniconda3/envs/spiralpp/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/students/u7043565/play/spiralpp/spiral-envs/setup.py'"'"'; file='"'"'/students/u7043565/play/spiralpp/spiral-envs/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.
Thank you for your time.