search

phoenix104104 / fast_blind_video_consistency

Learning Blind Video Temporal Consistency (ECCV 2018)
http://vllab.ucmerced.edu/wlai24/video_consistency/
Other
423 stars 64 forks source link

Compilation error #1

Closed amjltc295 closed 6 years ago

amjltc295 commented 6 years ago

Hello, it's a great project and I want to reproduce your work. However, I got some errors installing the FlowNet2:

$ ./install.sh 
Compiling correlation kernels by nvcc...
/home/allen/MiRA/fast_blind_video_consistency/networks/correlation_package/src/correlation_cuda.c: In function ‘Correlation_forward_cuda’:
/home/allen/MiRA/fast_blind_video_consistency/networks/correlation_package/src/correlation_cuda.c:20:25: error: dereferencing pointer to incomplete type ‘THTensor {aka struct THTensor}’
   int batchSize = input1->size[0];
                         ^
Traceback (most recent call last):
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/unixccompiler.py", line 118, in _compile
    extra_postargs)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/ccompiler.py", line 909, in spawn
    spawn(cmd, dry_run=self.dry_run)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/spawn.py", line 36, in spawn
    _spawn_posix(cmd, search_path, dry_run=dry_run)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/spawn.py", line 159, in _spawn_posix
    % (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/cffi/ffiplatform.py", line 51, in _build
    dist.run_command('build_ext')
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
    self._build_extensions_serial()
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/command/build_ext.py", line 533, in build_extension
    depends=ext.depends)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/unixccompiler.py", line 120, in _compile
    raise CompileError(msg)
distutils.errors.CompileError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "build.py", line 31, in <module>
    ffi.build()
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/torch/utils/ffi/__init__.py", line 189, in build
    _build_extension(ffi, cffi_wrapper_name, target_dir, verbose)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/torch/utils/ffi/__init__.py", line 111, in _build_extension
    outfile = ffi.compile(tmpdir=tmpdir, verbose=verbose, target=libname)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/cffi/api.py", line 697, in compile
    compiler_verbose=verbose, debug=debug, **kwds)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/cffi/recompiler.py", line 1520, in recompile
    compiler_verbose, debug)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/cffi/ffiplatform.py", line 22, in compile
    outputfilename = _build(tmpdir, ext, compiler_verbose, debug)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/cffi/ffiplatform.py", line 58, in _build
    raise VerificationError('%s: %s' % (e.__class__.__name__, e))
cffi.error.VerificationError: CompileError: command 'gcc' failed with exit status 1
Compiling resample2d kernels by nvcc...
rm: cannot remove 'Resample2d_kernel.o': No such file or directory
In file included from /home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/torch/lib/include/THC/THCGeneral.h:5:0,
                 from /home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/torch/lib/include/THC/THC.h:4,
                 from Resample2d_kernel.cu:1:
/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/torch/lib/include/TH/THAllocator.h:6:28: fatal error: ATen/Allocator.h: No such file or directory
compilation terminated.
gcc: error: /home/allen/MiRA/fast_blind_video_consistency/networks/resample2d_package/src/Resample2d_kernel.o: No such file or directory
Traceback (most recent call last):
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/unixccompiler.py", line 197, in link
    self.spawn(linker + ld_args)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/ccompiler.py", line 909, in spawn
    spawn(cmd, dry_run=self.dry_run)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/spawn.py", line 36, in spawn
    _spawn_posix(cmd, search_path, dry_run=dry_run)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/spawn.py", line 159, in _spawn_posix
    % (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/cffi/ffiplatform.py", line 51, in _build
    dist.run_command('build_ext')
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
    self._build_extensions_serial()
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/command/build_ext.py", line 558, in build_extension
    target_lang=language)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/ccompiler.py", line 717, in link_shared_object
    extra_preargs, extra_postargs, build_temp, target_lang)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/unixccompiler.py", line 199, in link
    raise LinkError(msg)
distutils.errors.LinkError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "build.py", line 31, in <module>
    ffi.build()
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/torch/utils/ffi/__init__.py", line 189, in build
    _build_extension(ffi, cffi_wrapper_name, target_dir, verbose)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/torch/utils/ffi/__init__.py", line 111, in _build_extension
    outfile = ffi.compile(tmpdir=tmpdir, verbose=verbose, target=libname)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/cffi/api.py", line 697, in compile
    compiler_verbose=verbose, debug=debug, **kwds)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/cffi/recompiler.py", line 1520, in recompile
    compiler_verbose, debug)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/cffi/ffiplatform.py", line 22, in compile
    outputfilename = _build(tmpdir, ext, compiler_verbose, debug)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/cffi/ffiplatform.py", line 58, in _build
    raise VerificationError('%s: %s' % (e.__class__.__name__, e))
cffi.error.VerificationError: LinkError: command 'gcc' failed with exit status 1
Compiling channelnorm kernels by nvcc...
rm: cannot remove 'ChannelNorm_kernel.o': No such file or directory
In file included from /home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/torch/lib/include/THC/THCGeneral.h:5:0,
                 from /home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/torch/lib/include/THC/THC.h:4,
                 from ChannelNorm_kernel.cu:1:
/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/torch/lib/include/TH/THAllocator.h:6:28: fatal error: ATen/Allocator.h: No such file or directory
compilation terminated.
gcc: error: /home/allen/MiRA/fast_blind_video_consistency/networks/channelnorm_package/src/ChannelNorm_kernel.o: No such file or directory
Traceback (most recent call last):
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/unixccompiler.py", line 197, in link
    self.spawn(linker + ld_args)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/ccompiler.py", line 909, in spawn
    spawn(cmd, dry_run=self.dry_run)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/spawn.py", line 36, in spawn
    _spawn_posix(cmd, search_path, dry_run=dry_run)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/spawn.py", line 159, in _spawn_posix
    % (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/cffi/ffiplatform.py", line 51, in _build
    dist.run_command('build_ext')
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
    self._build_extensions_serial()
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/command/build_ext.py", line 558, in build_extension
    target_lang=language)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/ccompiler.py", line 717, in link_shared_object
    extra_preargs, extra_postargs, build_temp, target_lang)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/distutils/unixccompiler.py", line 199, in link
    raise LinkError(msg)
distutils.errors.LinkError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "build.py", line 31, in <module>
    ffi.build()
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/torch/utils/ffi/__init__.py", line 189, in build
    _build_extension(ffi, cffi_wrapper_name, target_dir, verbose)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/torch/utils/ffi/__init__.py", line 111, in _build_extension
    outfile = ffi.compile(tmpdir=tmpdir, verbose=verbose, target=libname)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/cffi/api.py", line 697, in compile
    compiler_verbose=verbose, debug=debug, **kwds)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/cffi/recompiler.py", line 1520, in recompile
    compiler_verbose, debug)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/cffi/ffiplatform.py", line 22, in compile
    outputfilename = _build(tmpdir, ext, compiler_verbose, debug)
  File "/home/allen/miniconda3/envs/fast_blind_video_consistency/lib/python3.6/site-packages/cffi/ffiplatform.py", line 58, in _build
    raise VerificationError('%s: %s' % (e.__class__.__name__, e))
cffi.error.VerificationError: LinkError: command 'gcc' failed with exit status 1

Versions:

$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Tue_Jun_12_23:07:04_CDT_2018
Cuda compilation tools, release 9.2, V9.2.148
$ gcc --version
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ conda list
# packages in environment at /home/allen/miniconda3/envs/fast_blind_video_consistency:
#
# Name                    Version                   Build  Channel
blas                      1.0                         mkl  
ca-certificates           2018.03.07                    0  
certifi                   2018.4.16                py36_0  
cffi                      1.11.5           py36h9745a5d_0  
cudatoolkit               9.0                  h13b8566_0  
freetype                  2.9.1                h8a8886c_0  
intel-openmp              2018.0.3                      0  
jpeg                      9b                   h024ee3a_2  
libedit                   3.1.20170329         h6b74fdf_2  
libffi                    3.2.1                hd88cf55_4  
libgcc-ng                 7.2.0                hdf63c60_3  
libgfortran-ng            7.2.0                hdf63c60_3  
libpng                    1.6.34               hb9fc6fc_0  
libstdcxx-ng              7.2.0                hdf63c60_3  
libtiff                   4.0.9                he85c1e1_1  
mkl                       2018.0.3                      1  
mkl_fft                   1.0.4            py36h4414c95_1  
mkl_random                1.0.1            py36h4414c95_1  
ncurses                   6.1                  hf484d3e_0  
ninja                     1.8.2            py36h6bb024c_1  
numpy                     1.15.0           py36h1b885b7_0  
numpy-base                1.15.0           py36h3dfced4_0  
olefile                   0.45.1                   py36_0  
openssl                   1.0.2o               h14c3975_1  
pillow                    5.2.0            py36heded4f4_0  
pip                       10.0.1                   py36_0  
protobuf                  3.6.0                     <pip>
pycparser                 2.18                     py36_1  
python                    3.6.6                hc3d631a_0  
pytorch                   0.4.1           py36_cuda9.0.176_cudnn7.1.2_1    pytorch
readline                  7.0                  ha6073c6_4  
setuptools                39.2.0                   py36_0  
six                       1.11.0                   py36_1  
sqlite                    3.24.0               h84994c4_0  
tensorboardX              1.2                       <pip>
tk                        8.6.7                hc745277_3  
torchvision               0.2.1                    py36_1    pytorch
wheel                     0.31.1                   py36_0  
xz                        5.2.4                h14c3975_4  
zlib                      1.2.11               ha838bed_2

Did I miss something?

Allen

amjltc295 commented 6 years ago

Downgrade Pytorch to 0.4.0 seems to solve the compilation problem for me. Also, it needs CUDA-9.0 and does not work on CUDA-9.2.

phoenix104104 commented 6 years ago

Thanks for your report. I also double check that the correlation, channel norm, and resample2d packages cannot be compiled on Pytorch 0.4.1. Unfortunately, I am not familiar with cuda. You may want to open an issue on the original flownet2-pytorch repo.