search

uber-research / jpeg2dct

Other
258 stars 45 forks source link

error: command 'gcc' failed with exit status 1 #17

Open lchia opened 3 years ago

lchia commented 3 years ago

How to solve gcc failure ? Thanks!

**_$ python setup.py install_**
running install
running bdist_egg
running egg_info
creating jpeg2dct.egg-info
writing jpeg2dct.egg-info/PKG-INFO
~/anaconda3/envs/vsr/lib/python3.7/site-packages/setuptools/dist.py:125: UserWarning: newlines not allowed and will break in the future
  warnings.warn("newlines not allowed and will break in the future")
writing dependency_links to jpeg2dct.egg-info/dependency_links.txt
writing requirements to jpeg2dct.egg-info/requires.txt
writing top-level names to jpeg2dct.egg-info/top_level.txt
writing manifest file 'jpeg2dct.egg-info/SOURCES.txt'
reading manifest file 'jpeg2dct.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'jpeg2dct.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/test
copying test/__init__.py -> build/lib.linux-x86_64-3.7/test
creating build/lib.linux-x86_64-3.7/jpeg2dct
copying jpeg2dct/__init__.py -> build/lib.linux-x86_64-3.7/jpeg2dct
creating build/lib.linux-x86_64-3.7/test/numpy
copying test/numpy/test_load.py -> build/lib.linux-x86_64-3.7/test/numpy
copying test/numpy/__init__.py -> build/lib.linux-x86_64-3.7/test/numpy
creating build/lib.linux-x86_64-3.7/test/tensorflow
copying test/tensorflow/__init__.py -> build/lib.linux-x86_64-3.7/test/tensorflow
copying test/tensorflow/test_decode.py -> build/lib.linux-x86_64-3.7/test/tensorflow
creating build/lib.linux-x86_64-3.7/jpeg2dct/petastorm
copying jpeg2dct/petastorm/codecs.py -> build/lib.linux-x86_64-3.7/jpeg2dct/petastorm
copying jpeg2dct/petastorm/__init__.py -> build/lib.linux-x86_64-3.7/jpeg2dct/petastorm
creating build/lib.linux-x86_64-3.7/jpeg2dct/common
copying jpeg2dct/common/__init__.py -> build/lib.linux-x86_64-3.7/jpeg2dct/common
creating build/lib.linux-x86_64-3.7/jpeg2dct/numpy
copying jpeg2dct/numpy/__init__.py -> build/lib.linux-x86_64-3.7/jpeg2dct/numpy
copying jpeg2dct/numpy/dctfromjpg_wrapper.py -> build/lib.linux-x86_64-3.7/jpeg2dct/numpy
creating build/lib.linux-x86_64-3.7/jpeg2dct/tensorflow
copying jpeg2dct/tensorflow/__init__.py -> build/lib.linux-x86_64-3.7/jpeg2dct/tensorflow
running build_ext
gcc -pthread -B ~/anaconda3/envs/vsr/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwr                           apv -O3 -Wall -Wstrict-prototypes -fPIC -std=c++11 -fPIC -O2 -I~/anaconda3/envs/vsr/include/python3.                           7m -c build/temp.linux-x86_64-3.7/test_compile/test_cpp_flags.cc -o build/temp.linux-x86_64-3.7/test_compile/test_c                           pp_flags.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
gcc -pthread -shared -B ~/anaconda3/envs/vsr/compiler_compat -L~/anaconda3/envs/vsr/l                           ib -Wl,-rpath=~/anaconda3/envs/vsr/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.                           7/test_compile/test_cpp_flags.o -o build/temp.linux-x86_64-3.7/test_compile/test_cpp_flags.so
2021-10-22 17:13:45.985289: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic                            library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory;                            LD_LIBRARY_PATH: ~/local/libjpeg/lib64:~/local/libjpeg/bin:~/local/li                           bjpeg/bin:~/local/gcc-5.2.0/bin:~/ffmpeg-git-20210908-amd64-static/ffmpeg:/mnt/data/i                           sp001/cuda-10.2/bin:~/anaconda3/envs/vsr/bin:~/anaconda3/condabin:/usr/local/bin:/usr                           /bin:/usr/local/sbin:/usr/sbin:/usr/local/cuda-10.0/bin:~/.local/bin:~/bin
2021-10-22 17:13:45.985368: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you                            do not have a GPU set up on your machine.
building 'jpeg2dct.tensorflow.tf_lib' extension
creating build/temp.linux-x86_64-3.7/jpeg2dct
creating build/temp.linux-x86_64-3.7/jpeg2dct/tensorflow
gcc -pthread -B ~/anaconda3/envs/vsr/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwr                           apv -O3 -Wall -Wstrict-prototypes -fPIC -I~/anaconda3/envs/vsr/include -I~/anaconda3/                           envs/vsr/include/python3.7m -c jpeg2dct/tensorflow/tf_lib.cc -o build/temp.linux-x86_64-3.7/jpeg2dct/tensorflow/tf_                           lib.o -std=c++11 -fPIC -O2 -I~/anaconda3/envs/vsr/lib/python3.7/site-packages/tensorflow/include -D_                           GLIBCXX_USE_CXX11_ABI=0 -DEIGEN_MAX_ALIGN_BYTES=64
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
In file included from ~/anaconda3/envs/vsr/lib/python3.7/site-packages/tensorflow/include/tensorflow/core/framework/attr_value_util.h:24:0,
                 from ~/anaconda3/envs/vsr/lib/python3.7/site-packages/tensorflow/include/tensorflow/core/framework/node_def_util.h:23,
                 from ~/anaconda3/envs/vsr/lib/python3.7/site-packages/tensorflow/include/tensorflow/core/framework/full_type_util.h:24,
                 from ~/anaconda3/envs/vsr/lib/python3.7/site-packages/tensorflow/include/tensorflow/core/framework/op.h:24,
                 from jpeg2dct/tensorflow/tf_lib.cc:10:
~/anaconda3/envs/vsr/lib/python3.7/site-packages/tensorflow/include/tensorflow/core/framework/tensor.h: In instantiation of ‘typename tensorflow::TTypes<T>::ConstScalar tensorflow::Tensor::scalar() const [with T = std::basic_string<char>; typename tensorflow::TTypes<T>::ConstScalar = Eigen::TensorMap<Eigen::TensorFixedSize<const std::basic_string<char>, Eigen::Sizes<>, 1, long int>, 16, Eigen::MakePointer>]’:
jpeg2dct/tensorflow/tf_lib.cc:32:53:   required from here
~/anaconda3/envs/vsr/lib/python3.7/site-packages/tensorflow/include/tensorflow/core/framework/tensor.h:905:3: error: static assertion failed: std::string is no longer a scalar type, use tensorflow::tstring
   static_assert(
   ^
INFO: Unable to build TensorFlow plugin, will skip it.

Traceback (most recent call last):
  File "~/anaconda3/envs/vsr/lib/python3.7/distutils/unixccompiler.py", line 118, in _compile
    extra_postargs)
  File "~/anaconda3/envs/vsr/lib/python3.7/distutils/ccompiler.py", line 910, in spawn
    spawn(cmd, dry_run=self.dry_run)
  File "~/anaconda3/envs/vsr/lib/python3.7/distutils/spawn.py", line 36, in spawn
    _spawn_posix(cmd, search_path, dry_run=dry_run)
  File "~/anaconda3/envs/vsr/lib/python3.7/distutils/spawn.py", line 159, in _spawn_posix
    % (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "setup.py", line 270, in build_extensions
    abi_compile_flags = build_tf_extension(self, options)
  File "setup.py", line 255, in build_tf_extension
    build_ext.build_extension(tf_lib)
  File "~/anaconda3/envs/vsr/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 196, in build_extension
    _build_ext.build_extension(self, ext)
  File "~/anaconda3/envs/vsr/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension
    depends=ext.depends)
  File "~/anaconda3/envs/vsr/lib/python3.7/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "~/anaconda3/envs/vsr/lib/python3.7/distutils/unixccompiler.py", line 120, in _compile
    raise CompileError(msg)
distutils.errors.CompileError: command 'gcc' failed with exit status 1

building 'jpeg2dct.common.common_lib' extension
creating build/temp.linux-x86_64-3.7/jpeg2dct/common
gcc -pthread -B ~/anaconda3/envs/vsr/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I~/anaconda3/envs/vsr/include -I~/anaconda3/envs/vsr/include/python3.7m -c jpeg2dct/common/dctfromjpg.cc -o build/temp.linux-x86_64-3.7/jpeg2dct/common/dctfromjpg.o -std=c++11 -fPIC -O2
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
jpeg2dct/common/dctfromjpg.cc:16:21: fatal error: jpeglib.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1

**_$ gcc --version_**
gcc (GCC) 5.2.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
lchia commented 3 years ago

Install libjpeg & jpeg2dct

1. Download source

Websit: libjpeg

2. Install

cd ~/tools/
cd jpeg-9d/
mkdir ~/local/libjpeg
./configure --prefix ~/local/libjpeg
make -j32
make install

ls ~/local/libjpeg/
test -j32

3. Add environment

#######libjpeg
export PATH="~/local/libjpeg/bin:$PATH"
export LD_LIBRARY_PATH="~/local/libjpeg/lib64:$PATH"

3.2 如何给conda的env添加环境变量

Managing environments

cd $CONDA_PREFIX
mkdir -p ./etc/conda/activate.d
mkdir -p ./etc/conda/deactivate.d
touch ./etc/conda/activate.d/env_vars.sh
touch ./etc/conda/deactivate.d/env_vars.sh

### Edit ./etc/conda/activate.d/env_vars.sh as follows:

#!/bin/sh

export MY_KEY='secret-key-value'
export MY_FILE=/path/to/my/file/

### Edit ./etc/conda/deactivate.d/env_vars.sh as follows:

#!/bin/sh

unset MY_KEY
unset MY_FILE

4. Install jpeg2dct

git clone https://github.com/uber-research/jpeg2dct.git
cd jpeg2dct
python setup.py install

Need to modify setup.py to provide the libjpeg include & library path

def get_common_options(build_ext):
    cpp_flags = get_cpp_flags(build_ext)

    MACROS = []
    INCLUDES = ['~/local/libjpeg/include'] + get_conda_include_dir()
    SOURCES = []
    COMPILE_FLAGS = cpp_flags
    LINK_FLAGS = []
    LIBRARY_DIRS = ['~/local/libjpeg/lib']
    LIBRARIES = []

    return dict(MACROS=MACROS,
                INCLUDES=INCLUDES,
                SOURCES=SOURCES,
                COMPILE_FLAGS=COMPILE_FLAGS,
                LINK_FLAGS=LINK_FLAGS,
                LIBRARY_DIRS=LIBRARY_DIRS,
                LIBRARIES=LIBRARIES)

5. Reference

libjpeg install

lchia commented 3 years ago

Errors appear when my code use jpeg2dct ....

**_$ python main.py_**

Traceback (most recent call last):
  File "main.py", line 32, in <module>
    tudct = TransformUpscaledDCT()
  File "main.py", line 19, in __init__
    self.jpeg_encoder = TurboJPEG('/usr/lib/libturbojpeg.so')
  File "/mnt/data/isp001/anaconda3/envs/vsr/lib/python3.7/site-packages/turbojpeg.py", line 288, in __init__
    self.__find_turbojpeg() if lib_path is None else lib_path)
  File "~/anaconda3/envs/vsr/lib/python3.7/ctypes/__init__.py", line 442, in LoadLibrary
    return self._dlltype(name)
  File "~/anaconda3/envs/vsr/lib/python3.7/ctypes/__init__.py", line 364, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /usr/lib/libturbojpeg.so: cannot open shared object file: No such file or directory