Linking error when building on Google Collaboratory

sjscotti commented 5 years ago

Hi! I really am impressed with waifu2x on my Windows PC, but for converting videos I don't have enough resources. So I have been trying to use the free cloud computing capability of the Google Collaboratory (https://colab.research.google.com/), and following your installation instructions there. I was successful building Caffe from source, but because it is time consuming and the virtual machine disappears after each session, I am now installing the prebuilt Caffe application/library using.

%%shell
sudo apt update
sudo apt upgrade
sudo apt install caffe-cuda

(Note the %%shell is required because colab is based on Jupyter (python) notebooks.) I don't think the problem is Caffe. I am using the follow script to install waifu2x

%%shell
git clone --recursive -b ubuntu https://github.com/nagadomi/waifu2x-caffe.git
cd waifu2x-caffe
git submodule update --init --recursive

# add a bunch of libraries to try and fix the ld error (none seem to help)
sudo apt-get update
sudo apt-get install build-essential
sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev 
sudo apt-get install libboost-dev libboost-all-dev libhdf5-serial-dev libgflags-dev
sudo apt-get install libgoogle-glog-dev liblmdb-dev protobluf-compiler
sudo apt-get install libprotoc-dev libpthread-stubs0-dev 
sudo apt-get install python-dev python-pip gfortran
sudo apt-get -y install libboost-tools-dev libboost-thread-dev magics++

# create symlink to ltcggie-caffe
ln -s /usr/bin ./caffe
ln -s /usr/lib/x86_64-linux-gnu ./libcaffe

# build 
rm -fr build # clean
mkdir build
cd build
cmake .. -DCUDA_NVCC_FLAGS="-D_FORCE_INLINES  -gencode arch=compute_61,code=sm_61 " # sm_61 is for GTX1080
make
ln -s `realpath ./waifu2x-caffe` ../bin

And I get a loader error. The following is an extract from the waifu2x-caffe/build/CMakeFiles/CMakeError.log file...


Determining if the pthread_create exist failed with the following output:
Change Dir: /content/waifu2x-caffe/build/CMakeFiles/CMakeTmp

Run Build Command:"/usr/bin/make" "cmTC_8c73f/fast"
/usr/bin/make -f CMakeFiles/cmTC_8c73f.dir/build.make CMakeFiles/cmTC_8c73f.dir/build
make[1]: Entering directory '/content/waifu2x-caffe/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_8c73f.dir/CheckSymbolExists.c.o
/usr/bin/cc   -std=c++11 -fpermissive -Wunused-variable  -Wunused-variable     -o CMakeFiles/cmTC_8c73f.dir/CheckSymbolExists.c.o   -c /content/waifu2x-caffe/build/CMakeFiles/CMakeTmp/CheckSymbolExists.c
cc1: warning: command line option ‘-std=c++11’ is valid for C++/ObjC++ but not for C
cc1: warning: command line option ‘-fpermissive’ is valid for C++/ObjC++ but not for C
Linking C executable cmTC_8c73f
/usr/local/lib/python2.7/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/cmTC_8c73f.dir/link.txt --verbose=1
/usr/bin/cc  -std=c++11 -fpermissive -Wunused-variable  -Wunused-variable      -rdynamic CMakeFiles/cmTC_8c73f.dir/CheckSymbolExists.c.o  -o cmTC_8c73f 
CMakeFiles/cmTC_8c73f.dir/CheckSymbolExists.c.o: In function `main':
CheckSymbolExists.c:(.text+0x1b): undefined reference to `pthread_create'
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_8c73f.dir/build.make:86: recipe for target 'cmTC_8c73f' failed
make[1]: *** [cmTC_8c73f] Error 1
make[1]: Leaving directory '/content/waifu2x-caffe/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_8c73f/fast' failed
make: *** [cmTC_8c73f/fast] Error 2

File /content/waifu2x-caffe/build/CMakeFiles/CMakeTmp/CheckSymbolExists.c:
/* */
#include <pthread.h>

int main(int argc, char** argv)
{
  (void)argv;
#ifndef pthread_create
  return ((int*)(&pthread_create))[argc];
#else
  (void)argc;
  return 0;
#endif
}

Determining if the function pthread_create exists in the pthreads failed with the following output:
Change Dir: /content/waifu2x-caffe/build/CMakeFiles/CMakeTmp

Run Build Command:"/usr/bin/make" "cmTC_70d95/fast"
/usr/bin/make -f CMakeFiles/cmTC_70d95.dir/build.make CMakeFiles/cmTC_70d95.dir/build
make[1]: Entering directory '/content/waifu2x-caffe/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_70d95.dir/CheckFunctionExists.c.o
/usr/bin/cc   -std=c++11 -fpermissive -Wunused-variable  -Wunused-variable  -DCHECK_FUNCTION_EXISTS=pthread_create   -o CMakeFiles/cmTC_70d95.dir/CheckFunctionExists.c.o   -c /usr/local/lib/python2.7/dist-packages/cmake/data/share/cmake-3.12/Modules/CheckFunctionExists.c
cc1: warning: command line option ‘-std=c++11’ is valid for C++/ObjC++ but not for C
cc1: warning: command line option ‘-fpermissive’ is valid for C++/ObjC++ but not for C
Linking C executable cmTC_70d95
/usr/local/lib/python2.7/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/cmTC_70d95.dir/link.txt --verbose=1
/usr/bin/cc  -std=c++11 -fpermissive -Wunused-variable  -Wunused-variable  -DCHECK_FUNCTION_EXISTS=pthread_create    -rdynamic CMakeFiles/cmTC_70d95.dir/CheckFunctionExists.c.o  -o cmTC_70d95 -lpthreads 
/usr/bin/ld: cannot find -lpthreads
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_70d95.dir/build.make:86: recipe for target 'cmTC_70d95' failed
make[1]: *** [cmTC_70d95] Error 1
make[1]: Leaving directory '/content/waifu2x-caffe/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_70d95/fast' failed
make: *** [cmTC_70d95/fast] Error 2

I have been trying for several days to address the /usr/bin/ld: cannot find -lpthreads error based on what I can find out about it on the internet. I added a number of sudo apt-get ... lines to your build script, but none made a difference.
Do you have any suggestions on how to fix this build problem? Thanks in advance!

nagadomi commented 5 years ago

building this code requires lltcggie's modified caffe. caffe-cuda installed from apt will not work.

sjscotti commented 5 years ago

Thanks for that info! I found I had to update cmake and to build lltcggie caffe I had to add a number of apt calls to install dependencies. Here is the script I used to build caffe...

# apt calls required to install dependencies needed to build caffe library
apt update
apt install g++-5
sudo apt-get install dialog # fixes "debconf: unable to initialize frontend: Dialog..." error
sudo apt-get install libprotobuf-dev protobuf-c-compiler protobuf-compiler 
sudo apt-get install libgflags-dev # from https://github.com/BVLC/caffe/wiki/Commonly-encountered-build-issues
sudo apt-get install libgoogle-glog-dev # from https://github.com/BVLC/caffe/wiki/Commonly-encountered-build-issues
sudo apt-get install liblmdb-dev # from https://github.com/BVLC/caffe/wiki/Commonly-encountered-build-issues
sudo apt-get install libleveldb-dev # from https://gist.github.com/dustismo/6203329#file-gistfile1-sh
sudo apt-get install libsnappy-dev # from https://github.com/BVLC/caffe/issues/4490
git clone -b waifu2x-caffe-ubuntu https://github.com/nagadomi/caffe.git lltcggie-caffe
cd lltcggie-caffe
cp /content/Makefile.config .
# Makefile.config was edited to remove -gencode arch=compute_20,code=sm_21 and placed on /content
make

Then I used your script to build waifu2x-caffe, but it did not go to completion. This is the output of the build process...

Cloning into 'waifu2x-caffe'...
remote: Enumerating objects: 2238, done.
remote: Total 2238 (delta 0), reused 0 (delta 0), pack-reused 2238
Receiving objects: 100% (2238/2238), 207.59 MiB | 12.86 MiB/s, done.
Resolving deltas: 100% (1380/1380), done.
Checking out files: 100% (165/165), done.
Submodule 'msgpack-c' (https://github.com/msgpack/msgpack-c) registered for path 'msgpack-c'
Submodule 'rapidjson' (https://github.com/miloyip/rapidjson.git) registered for path 'rapidjson'
Submodule 'stb' (https://github.com/nothings/stb.git) registered for path 'stb'
Cloning into '/content/waifu2x-caffe/msgpack-c'...
Cloning into '/content/waifu2x-caffe/rapidjson'...
Cloning into '/content/waifu2x-caffe/stb'...
Submodule path 'msgpack-c': checked out '42181289ed9c12fbf70ed3f1320c7ef9a4f76836'
Submodule 'external/boost/predef' (https://github.com/boostorg/predef.git) registered for path 'msgpack-c/external/boost/predef'
Submodule 'external/boost/preprocessor' (https://github.com/boostorg/preprocessor.git) registered for path 'msgpack-c/external/boost/preprocessor'
Cloning into '/content/waifu2x-caffe/msgpack-c/external/boost/predef'...
Cloning into '/content/waifu2x-caffe/msgpack-c/external/boost/preprocessor'...
Submodule path 'msgpack-c/external/boost/predef': checked out '1e8d1c2cfcac343305e2182f665742b9e38f7452'
Submodule path 'msgpack-c/external/boost/preprocessor': checked out 'c2beb74b90969f369c62d0dea68aa5f0e203268e'
Submodule path 'rapidjson': checked out 'c8c8ad47c372dc724cbabcbaf5c62aee4618afeb'
Submodule 'thirdparty/gtest' (https://chromium.googlesource.com/external/googletest.git) registered for path 'rapidjson/thirdparty/gtest'
Cloning into '/content/waifu2x-caffe/rapidjson/thirdparty/gtest'...
Submodule path 'rapidjson/thirdparty/gtest': checked out '0476e154db5fab1721c2a0f32abf4aa773679b52'
Submodule path 'stb': checked out '955dfe991b26f6fb1287ec0093f606843487b099'
-- The C compiler identification is GNU 7.4.0
-- The CXX compiler identification is GNU 7.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Warning (dev) at caffe/cmake/Misc.cmake:32 (set):
  implicitly converting 'BOOLEAN' to 'STRING' type.
Call Stack (most recent call first):
  caffe/CMakeLists.txt:24 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Boost version: 1.65.1
-- Found the following Boost libraries:
--   system
--   thread
--   filesystem
--   chrono
--   date_time
--   atomic
-- Found GFlags: /usr/include  
-- Found gflags  (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libgflags.so)
-- Found Glog: /usr/include  
-- Found glog    (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libglog.so)
-- Found Protobuf: /usr/lib/x86_64-linux-gnu/libprotobuf.so;-lpthread (found version "3.0.0") 
-- Found PROTOBUF Compiler: /usr/bin/protoc
-- HDF5: Using hdf5 compiler wrapper to determine C configuration
-- HDF5: Using hdf5 compiler wrapper to determine CXX configuration
-- Found HDF5: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5_cpp.so;/usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so;/usr/lib/x86_64-linux-gnu/libpthread.so;/usr/lib/x86_64-linux-gnu/libsz.so;/usr/lib/x86_64-linux-gnu/libz.so;/usr/lib/x86_64-linux-gnu/libdl.so;/usr/lib/x86_64-linux-gnu/libm.so (found version "1.10.0.1") found components:  HL 
-- Found LMDB: /usr/include  
-- Found lmdb    (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/liblmdb.so)
-- Found LevelDB: /usr/include  
-- Found LevelDB (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libleveldb.so)
-- Found Snappy: /usr/include  
-- Found Snappy  (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libsnappy.so)
-- CUDA detected: 10.0
-- Found cuDNN: ver. 7.6.3 found (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libcudnn.so)
-- Added CUDA NVCC flags for: sm_37
-- OpenCV found (/usr/share/OpenCV)
-- Found Atlas: /usr/include/x86_64-linux-gnu  
-- Found Atlas (include: /usr/include/x86_64-linux-gnu, library: /usr/lib/x86_64-linux-gnu/libatlas.so)
-- Found PythonInterp: /usr/bin/python2.7 (found suitable version "2.7.15", minimum required is "2.7") 
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython2.7.so (found suitable version "2.7.15+", minimum required is "2.7") 
-- Found NumPy: /usr/local/lib/python2.7/dist-packages/numpy/core/include (found suitable version "1.16.4", minimum required is "1.7.1") 
-- NumPy ver. 1.16.4 found (include: /usr/local/lib/python2.7/dist-packages/numpy/core/include)
-- Boost version: 1.65.1
-- Found the following Boost libraries:
--   python
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) 
-- Found Git: /usr/bin/git (found version "2.17.1") 
-- 
-- ******************* Caffe Configuration Summary *******************
-- General:
--   Version           :   1.0.0-rc3
--   Git               :   rc3-226-g3cca40e7
--   System            :   Linux
--   C++ compiler      :   /usr/bin/c++
--   Release CXX flags :   -O3 -DNDEBUG -std=c++11 -fpermissive -Wunused-variable  -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
--   Debug CXX flags   :   -g -std=c++11 -fpermissive -Wunused-variable  -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
--   Build type        :   Release
-- 
--   BUILD_SHARED_LIBS :   ON
--   BUILD_python      :   ON
--   BUILD_matlab      :   OFF
--   BUILD_docs        :   ON
--   CPU_ONLY          :   OFF
--   USE_OPENCV        :   ON
--   USE_LEVELDB       :   ON
--   USE_LMDB          :   ON
--   ALLOW_LMDB_NOLOCK :   OFF
-- 
-- Dependencies:
--   BLAS              :   Yes (Atlas)
--   Boost             :   Yes (ver. 1.65)
--   glog              :   Yes
--   gflags            :   Yes
--   protobuf          :   Yes (ver. 3.0.0)
--   lmdb              :   Yes (ver. 0.9.21)
--   LevelDB           :   Yes (ver. 1.20)
--   Snappy            :   Yes (ver. ..)
--   OpenCV            :   Yes (ver. 3.2.0)
--   CUDA              :   Yes (ver. 10.0)
-- 
-- NVIDIA CUDA:
--   Target GPU(s)     :   Auto
--   GPU arch(s)       :   sm_37
--   cuDNN             :   Yes (ver. 7.6.3)
-- 
-- Python:
--   Interpreter       :   /usr/bin/python2.7 (ver. 2.7.15)
--   Libraries         :   /usr/lib/x86_64-linux-gnu/libpython2.7.so (ver 2.7.15+)
--   NumPy             :   /usr/local/lib/python2.7/dist-packages/numpy/core/include (ver 1.16.4)
-- 
-- Documentaion:
--   Doxygen           :   No
--   config_file       :   
-- 
-- Install:
--   Install path      :   /content/waifu2x-caffe/build/libcaffe/install
-- 
-- Found cuDNN (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libcudnn.so)
-- Found cuDNN (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libcudnn.so)
-- Found OpenCV: /usr (found version "3.2.0") found components:  core imgproc highgui 
-- Boost version: 1.65.1
-- Found the following Boost libraries:
--   filesystem
--   system
--   iostreams
--   regex
-- Found GLOG: /usr/include  
-- Found CUDA: /usr/local/cuda (found version "10.0") 
-- Configuring done
-- Generating done
-- Build files have been written to: /content/waifu2x-caffe/build
[  0%] Running C++/Python protocol buffer compiler on /content/waifu2x-caffe/caffe/src/caffe/proto/caffe.proto
Scanning dependencies of target proto
[  1%] Building CXX object libcaffe/src/caffe/CMakeFiles/proto.dir/__/__/include/caffe/proto/caffe.pb.cc.o
[  1%] Linking CXX static library ../../lib/libproto.a
[  1%] Built target proto
[  1%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/util/cuda_compile_1_generated_math_functions.cu.o
In file included from /content/waifu2x-caffe/caffe/src/caffe/util/math_functions.cu:1:0:
/usr/local/cuda/include/math_functions.h:54:2: warning: #warning "math_functions.h is an internal header file and must not be used directly.  This file will be removed in a future CUDA release.  Please use cuda_runtime_api.h or cuda_runtime.h instead." [-Wcpp]
 #warning "math_functions.h is an internal header file and must not be used directly.  This file will be removed in a future CUDA release.  Please use cuda_runtime_api.h or cuda_runtime.h instead."
  ^~~~~~~
In file included from /content/waifu2x-caffe/caffe/src/caffe/util/math_functions.cu:1:0:
/usr/local/cuda/include/math_functions.h:54:2: warning: #warning "math_functions.h is an internal header file and must not be used directly.  This file will be removed in a future CUDA release.  Please use cuda_runtime_api.h or cuda_runtime.h instead." [-Wcpp]
 #warning "math_functions.h is an internal header file and must not be used directly.  This file will be removed in a future CUDA release.  Please use cuda_runtime_api.h or cuda_runtime.h instead."
  ^~~~~~~
In file included from /content/waifu2x-caffe/caffe/src/caffe/util/math_functions.cu:1:0:
/usr/local/cuda/include/math_functions.h:54:2: warning: #warning "math_functions.h is an internal header file and must not be used directly.  This file will be removed in a future CUDA release.  Please use cuda_runtime_api.h or cuda_runtime.h instead." [-Wcpp]
 #warning "math_functions.h is an internal header file and must not be used directly.  This file will be removed in a future CUDA release.  Please use cuda_runtime_api.h or cuda_runtime.h instead."
  ^~~~~~~
In file included from /content/waifu2x-caffe/caffe/src/caffe/util/math_functions.cu:1:0:
/usr/local/cuda/include/math_functions.h:54:2: warning: #warning "math_functions.h is an internal header file and must not be used directly.  This file will be removed in a future CUDA release.  Please use cuda_runtime_api.h or cuda_runtime.h instead." [-Wcpp]
 #warning "math_functions.h is an internal header file and must not be used directly.  This file will be removed in a future CUDA release.  Please use cuda_runtime_api.h or cuda_runtime.h instead."
  ^~~~~~~
In file included from /content/waifu2x-caffe/caffe/src/caffe/util/math_functions.cu:1:0:
/usr/local/cuda/include/math_functions.h:54:2: warning: #warning "math_functions.h is an internal header file and must not be used directly.  This file will be removed in a future CUDA release.  Please use cuda_runtime_api.h or cuda_runtime.h instead." [-Wcpp]
 #warning "math_functions.h is an internal header file and must not be used directly.  This file will be removed in a future CUDA release.  Please use cuda_runtime_api.h or cuda_runtime.h instead."
  ^~~~~~~
In file included from /content/waifu2x-caffe/caffe/src/caffe/util/math_functions.cu:1:0:
/usr/local/cuda/include/math_functions.h:54:2: warning: #warning "math_functions.h is an internal header file and must not be used directly.  This file will be removed in a future CUDA release.  Please use cuda_runtime_api.h or cuda_runtime.h instead." [-Wcpp]
 #warning "math_functions.h is an internal header file and must not be used directly.  This file will be removed in a future CUDA release.  Please use cuda_runtime_api.h or cuda_runtime.h instead."
  ^~~~~~~
[  1%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_absval_layer.cu.o
[  1%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_base_data_layer.cu.o
[  2%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_batch_norm_layer.cu.o
[  2%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_batch_reindex_layer.cu.o
[  2%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_bias_layer.cu.o
[  4%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_bnll_layer.cu.o
[  4%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_concat_layer.cu.o
[  4%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_contrastive_loss_layer.cu.o
[  5%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_conv_layer.cu.o
[  5%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_crop_layer.cu.o
[  5%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_cudnn_conv_layer.cu.o
[  6%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_cudnn_deconv_layer.cu.o
[  6%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_cudnn_lcn_layer.cu.o
[  8%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_cudnn_lrn_layer.cu.o
[  8%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_cudnn_pooling_layer.cu.o
[  8%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_cudnn_relu_layer.cu.o
[  9%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_cudnn_sigmoid_layer.cu.o
[  9%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_cudnn_softmax_layer.cu.o
[  9%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_cudnn_tanh_layer.cu.o
[ 10%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_deconv_layer.cu.o
[ 10%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_dropout_layer.cu.o
[ 10%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_eltwise_layer.cu.o
[ 12%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_elu_layer.cu.o
[ 12%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_embed_layer.cu.o
[ 12%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_euclidean_loss_layer.cu.o
[ 13%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_exp_layer.cu.o
[ 13%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_filter_layer.cu.o
[ 14%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_hdf5_data_layer.cu.o
[ 14%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_hdf5_output_layer.cu.o
[ 14%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_im2col_layer.cu.o
[ 16%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_inner_product_layer.cu.o
[ 16%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_log_layer.cu.o
[ 16%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_lrn_layer.cu.o
[ 17%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_lstm_unit_layer.cu.o
[ 17%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_mvn_layer.cu.o
[ 17%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_pooling_layer.cu.o
[ 18%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_power_layer.cu.o
[ 18%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_prelu_layer.cu.o
[ 18%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_recurrent_layer.cu.o
[ 20%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_reduction_layer.cu.o
[ 20%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_relu_layer.cu.o
[ 21%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_scale_layer.cu.o
[ 21%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_sigmoid_cross_entropy_loss_layer.cu.o
[ 21%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_sigmoid_layer.cu.o
[ 22%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_silence_layer.cu.o
[ 22%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_slice_layer.cu.o
[ 22%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_softmax_layer.cu.o
[ 24%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_softmax_loss_layer.cu.o
[ 24%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_split_layer.cu.o
[ 24%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_tanh_layer.cu.o
[ 25%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_threshold_layer.cu.o
[ 25%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/cuda_compile_1_generated_tile_layer.cu.o
[ 25%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/solvers/cuda_compile_1_generated_adadelta_solver.cu.o
[ 27%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/solvers/cuda_compile_1_generated_adagrad_solver.cu.o
[ 27%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/solvers/cuda_compile_1_generated_adam_solver.cu.o
[ 28%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/solvers/cuda_compile_1_generated_nesterov_solver.cu.o
[ 28%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/solvers/cuda_compile_1_generated_rmsprop_solver.cu.o
[ 28%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/solvers/cuda_compile_1_generated_sgd_solver.cu.o
[ 29%] Building NVCC (Device) object libcaffe/src/caffe/CMakeFiles/cuda_compile_1.dir/util/cuda_compile_1_generated_im2col.cu.o
Scanning dependencies of target caffe
[ 29%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/blob.cpp.o
[ 31%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/common.cpp.o
[ 31%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/data_reader.cpp.o
[ 31%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/data_transformer.cpp.o
[ 32%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/internal_thread.cpp.o
[ 32%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layer.cpp.o
[ 33%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layer_factory.cpp.o
[ 33%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/absval_layer.cpp.o
[ 33%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/accuracy_layer.cpp.o
[ 35%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/argmax_layer.cpp.o
[ 35%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/base_conv_layer.cpp.o
[ 35%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/base_data_layer.cpp.o
[ 36%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/batch_norm_layer.cpp.o
[ 36%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/batch_reindex_layer.cpp.o
[ 36%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/bias_layer.cpp.o
[ 37%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/bnll_layer.cpp.o
[ 37%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/concat_layer.cpp.o
[ 37%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/contrastive_loss_layer.cpp.o
[ 39%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/conv_layer.cpp.o
[ 39%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/crop_layer.cpp.o
[ 40%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/cudnn_conv_layer.cpp.o
[ 40%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/cudnn_deconv_layer.cpp.o
[ 40%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/cudnn_lcn_layer.cpp.o
[ 41%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/cudnn_lrn_layer.cpp.o
[ 41%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/cudnn_pooling_layer.cpp.o
[ 41%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/cudnn_relu_layer.cpp.o
[ 43%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/cudnn_sigmoid_layer.cpp.o
[ 43%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/cudnn_softmax_layer.cpp.o
[ 43%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/cudnn_tanh_layer.cpp.o
[ 44%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/data_layer.cpp.o
[ 44%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/deconv_layer.cpp.o
[ 44%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/dropout_layer.cpp.o
[ 45%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/dummy_data_layer.cpp.o
[ 45%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/eltwise_layer.cpp.o
[ 47%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/elu_layer.cpp.o
[ 47%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/embed_layer.cpp.o
[ 47%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/euclidean_loss_layer.cpp.o
[ 48%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/exp_layer.cpp.o
[ 48%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/filter_layer.cpp.o
[ 48%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/flatten_layer.cpp.o
[ 50%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/hdf5_data_layer.cpp.o
[ 50%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/hdf5_output_layer.cpp.o
[ 50%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/hinge_loss_layer.cpp.o
[ 51%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/im2col_layer.cpp.o
[ 51%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/image_data_layer.cpp.o
[ 51%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/infogain_loss_layer.cpp.o
[ 52%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/inner_product_layer.cpp.o
[ 52%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/input_layer.cpp.o
[ 54%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/log_layer.cpp.o
[ 54%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/loss_layer.cpp.o
[ 54%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/lrn_layer.cpp.o
[ 55%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/lstm_layer.cpp.o
[ 55%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/lstm_unit_layer.cpp.o
[ 55%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/memory_data_layer.cpp.o
[ 56%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/multinomial_logistic_loss_layer.cpp.o
[ 56%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/mvn_layer.cpp.o
[ 56%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/neuron_layer.cpp.o
[ 58%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/parameter_layer.cpp.o
[ 58%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/pooling_layer.cpp.o
[ 59%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/power_layer.cpp.o
[ 59%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/prelu_layer.cpp.o
[ 59%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/recurrent_layer.cpp.o
[ 60%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/reduction_layer.cpp.o
[ 60%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/relu_layer.cpp.o
[ 60%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/reshape_layer.cpp.o
[ 62%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/rnn_layer.cpp.o
[ 62%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/scale_layer.cpp.o
[ 62%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/sigmoid_cross_entropy_loss_layer.cpp.o
[ 63%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/sigmoid_layer.cpp.o
[ 63%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/silence_layer.cpp.o
[ 63%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/slice_layer.cpp.o
[ 64%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/softmax_layer.cpp.o
[ 64%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/softmax_loss_layer.cpp.o
[ 66%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/split_layer.cpp.o
[ 66%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/spp_layer.cpp.o
[ 66%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/tanh_layer.cpp.o
[ 67%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/threshold_layer.cpp.o
[ 67%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/tile_layer.cpp.o
[ 67%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/layers/window_data_layer.cpp.o
[ 68%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/net.cpp.o
[ 68%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/parallel.cpp.o
[ 68%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/solver.cpp.o
[ 70%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/solvers/adadelta_solver.cpp.o
[ 70%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/solvers/adagrad_solver.cpp.o
[ 70%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/solvers/adam_solver.cpp.o
[ 71%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/solvers/nesterov_solver.cpp.o
[ 71%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/solvers/rmsprop_solver.cpp.o
[ 72%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/solvers/sgd_solver.cpp.o
[ 72%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/syncedmem.cpp.o
[ 72%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/util/benchmark.cpp.o
[ 74%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/util/blocking_queue.cpp.o
[ 74%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/util/cudnn.cpp.o
[ 74%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/util/cudnn_func.cpp.o
[ 75%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/util/db.cpp.o
[ 75%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/util/db_leveldb.cpp.o
[ 75%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/util/db_lmdb.cpp.o
[ 77%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/util/hdf5.cpp.o
[ 77%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/util/im2col.cpp.o
[ 77%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/util/insert_splits.cpp.o
[ 78%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/util/io.cpp.o
[ 78%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/util/math_functions.cpp.o
[ 79%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/util/signal_handler.cpp.o
[ 79%] Building CXX object libcaffe/src/caffe/CMakeFiles/caffe.dir/util/upgrade_proto.cpp.o
[ 79%] Linking CXX shared library ../../lib/libcaffe.so
[ 79%] Built target caffe
Scanning dependencies of target waifu2x-caffe
[ 79%] Building CXX object CMakeFiles/waifu2x-caffe.dir/waifu2x-caffe/Source.cpp.o

---------------------------------------------------------------------------

UnicodeDecodeError                        Traceback (most recent call last)

<ipython-input-4-c315550295cb> in <module>()
----> 1 get_ipython().run_cell_magic('shell', '', 'cd /content\ngit clone -b ubuntu https://github.com/nagadomi/waifu2x-caffe.git\ncd waifu2x-caffe\ngit submodule update --init --recursive\n\n# create symlink to ltcggie-caffe\nln -s ../lltcggie-caffe ./caffe\nln -s ../lltcggie-caffe ./libcaffe\n\n# build\nrm -fr build # clean\nmkdir build\ncd build\ncmake .. -DCUDA_NVCC_FLAGS="-D_FORCE_INLINES  -gencode arch=compute_61,code=sm_61 " # sm_61 is for GTX1080\nmake\nln -s `realpath ./waifu2x-caffe` ../bin')

5 frames

/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py in run_cell_magic(self, magic_name, line, cell)
   2115             magic_arg_s = self.var_expand(line, stack_depth)
   2116             with self.builtin_trap:
-> 2117                 result = fn(magic_arg_s, cell)
   2118             return result
   2119 

/usr/local/lib/python3.6/dist-packages/google/colab/_system_commands.py in _shell_cell_magic(args, cmd)
    108   parsed_args = magic_arguments.parse_argstring(_shell_cell_magic, args)
    109 
--> 110   result = _run_command(cmd, clear_streamed_output=False)
    111   if not parsed_args.ignore_errors:
    112     result.check_returncode()

/usr/local/lib/python3.6/dist-packages/google/colab/_system_commands.py in _run_command(cmd, clear_streamed_output)
    193       os.close(child_pty)
    194 
--> 195       return _monitor_process(parent_pty, epoll, p, cmd, update_stdin_widget)
    196   finally:
    197     epoll.close()

/usr/local/lib/python3.6/dist-packages/google/colab/_system_commands.py in _monitor_process(parent_pty, epoll, p, cmd, update_stdin_widget)
    220   while True:
    221     try:
--> 222       result = _poll_process(parent_pty, epoll, p, cmd, decoder, state)
    223       if result is not None:
    224         return result

/usr/local/lib/python3.6/dist-packages/google/colab/_system_commands.py in _poll_process(parent_pty, epoll, p, cmd, decoder, state)
    273       output_available = True
    274       raw_contents = os.read(parent_pty, _PTY_READ_MAX_BYTES_FOR_TEST)
--> 275       decoded_contents = decoder.decode(raw_contents)
    276 
    277       sys.stdout.write(decoded_contents)

/usr/lib/python3.6/codecs.py in decode(self, input, final)
    319         # decode input (taking the buffer into account)
    320         data = self.buffer + input
--> 321         (result, consumed) = self._buffer_decode(data, self.errors, final)
    322         # keep undecoded input until the next call
    323         self.buffer = data[consumed:]

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x83 in position 280: invalid start byte

Any suggestion on what I can do to complete the build? Thanks in advance!

nagadomi commented 5 years ago

Maybe an error has occurred at waifu2x-caffe/Source.cpp, but it is not displayed due to ipython's error. See CMakeError.log.

Or in case of ipython error only, it is because Shift JIS is used in the source code. (a compiler warning message may contain Shift JIS code.) I've never used jupyter notebook so I don't know how to fix it.

sjscotti commented 5 years ago

here is the CMakeError.log output...

Determining if the pthread_create exist failed with the following output:
Change Dir: /content/waifu2x-caffe/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_cd829/fast 
/usr/bin/make -f CMakeFiles/cmTC_cd829.dir/build.make CMakeFiles/cmTC_cd829.dir/build
make[1]: Entering directory '/content/waifu2x-caffe/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_cd829.dir/CheckSymbolExists.c.o
/usr/bin/cc   -std=c++11 -fpermissive -Wunused-variable  -Wunused-variable     -o CMakeFiles/cmTC_cd829.dir/CheckSymbolExists.c.o   -c /content/waifu2x-caffe/build/CMakeFiles/CMakeTmp/CheckSymbolExists.c
cc1: warning: command line option ‘-std=c++11’ is valid for C++/ObjC++ but not for C
cc1: warning: command line option ‘-fpermissive’ is valid for C++/ObjC++ but not for C
Linking C executable cmTC_cd829
/usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/cmTC_cd829.dir/link.txt --verbose=1
/usr/bin/cc  -std=c++11 -fpermissive -Wunused-variable  -Wunused-variable      -rdynamic CMakeFiles/cmTC_cd829.dir/CheckSymbolExists.c.o  -o cmTC_cd829 
CMakeFiles/cmTC_cd829.dir/CheckSymbolExists.c.o: In function `main':
CheckSymbolExists.c:(.text+0x1b): undefined reference to `pthread_create'
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_cd829.dir/build.make:86: recipe for target 'cmTC_cd829' failed
make[1]: *** [cmTC_cd829] Error 1
make[1]: Leaving directory '/content/waifu2x-caffe/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_cd829/fast' failed
make: *** [cmTC_cd829/fast] Error 2

File /content/waifu2x-caffe/build/CMakeFiles/CMakeTmp/CheckSymbolExists.c:
/* */
#include <pthread.h>

int main(int argc, char** argv)
{
  (void)argv;
#ifndef pthread_create
  return ((int*)(&pthread_create))[argc];
#else
  (void)argc;
  return 0;
#endif
}

Determining if the function pthread_create exists in the pthreads failed with the following output:
Change Dir: /content/waifu2x-caffe/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_71a1d/fast 
/usr/bin/make -f CMakeFiles/cmTC_71a1d.dir/build.make CMakeFiles/cmTC_71a1d.dir/build
make[1]: Entering directory '/content/waifu2x-caffe/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_71a1d.dir/CheckFunctionExists.c.o
/usr/bin/cc   -std=c++11 -fpermissive -Wunused-variable  -Wunused-variable  -DCHECK_FUNCTION_EXISTS=pthread_create   -o CMakeFiles/cmTC_71a1d.dir/CheckFunctionExists.c.o   -c /usr/local/lib/python3.6/dist-packages/cmake/data/share/cmake-3.14/Modules/CheckFunctionExists.c
cc1: warning: command line option ‘-std=c++11’ is valid for C++/ObjC++ but not for C
cc1: warning: command line option ‘-fpermissive’ is valid for C++/ObjC++ but not for C
Linking C executable cmTC_71a1d
/usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/cmTC_71a1d.dir/link.txt --verbose=1
/usr/bin/cc  -std=c++11 -fpermissive -Wunused-variable  -Wunused-variable  -DCHECK_FUNCTION_EXISTS=pthread_create    -rdynamic CMakeFiles/cmTC_71a1d.dir/CheckFunctionExists.c.o  -o cmTC_71a1d -lpthreads 
/usr/bin/ld: cannot find -lpthreads
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_71a1d.dir/build.make:86: recipe for target 'cmTC_71a1d' failed
make[1]: *** [cmTC_71a1d] Error 1
make[1]: Leaving directory '/content/waifu2x-caffe/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_71a1d/fast' failed
make: *** [cmTC_71a1d/fast] Error 2

The error log file give the same error as when I used the prebuilt caffe library (i.e., it could not find pthread_create in pthreads). However, the cmake output showed...

-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Boost version: 1.65.1
-- Found the following Boost libraries:
--   system
--   thread
--   filesystem
--   chrono
--   date_time
--   atomic

So it appears it found pthread_create in pthread. So I guess that isn't an issue.

I also looked to see if Source.cpp compiled and I found this file when doing an ls on /content/waifu2x-caffe/build/CMakeFiles/waifu2x-caffe.dir/waifu2x-caffe... -rw-r--r-- 1 root root 962296 Aug 31 13:04 Source.cpp.o

Does the file size look as you would expect? If so, it appears that Source.cpp compiled OK. Do you have any suggestions? Is there something I can do to localize where the build failed and why it failed? Thanks in advance!

sjscotti commented 5 years ago

Hi again! In thinking about this, perhaps the issue contained in your comment...

Or in case of ipython error only, it is because Shift JIS is used in the source code. (a compiler warning message may contain Shift JIS code.)

To correct this, would I need to edit the Source.cpp file to eliminate these characters?

sjscotti commented 5 years ago

Hi! That last suggestion was it! I had to change all the text - including comments - with Japanese characters to English (I used google translate). There was the Source.cpp file and also all 6 files under the common directory. I've only run the executable with the --help flag so far, but that worked. Now I'll need to see if it does conversions correctly. Thanks for your help!

sjscotti commented 5 years ago

Hi again! I successfully tested a single image and it appears to work for option -p gpu but not for -p cudnn.

nagadomi commented 5 years ago

It seems to be a character code handling issue in Jupyter notebook. waifu2x-caffe-cli is not fully internationalized and uses Japanese multibyte encoding called Shift JIS (or CP932). It can be converted to UTF-8 using nkf command. (or just replace it with English).

When using cuDNN, cuDNN header file and lib must be placed in CUDA_INCLUDE_DIR/CUDA_LIB_DIR, and set USE_CUDNN: = 1 in Makefile.conf before building caffe.

sjscotti commented 5 years ago

Thanks! I've also added the cuDNN header file and lib in building caffe and it appears to be working! I'll close this out, but I have another question that I'll start in a new thread.

IEWbgfnYDwHRoRRSKtkdyMDUzgdwuBYgDKtDJWd commented 4 years ago

Thanks! I've also added the cuDNN header file and lib in building caffe and it appears to be working! I'll close this out, but I have another question that I'll start in a new thread.

any chance you could share the working notebook ..?

LordBlackout commented 4 years ago

Yes, please share the working notebook!

mikebilly commented 2 years ago

can you please share the working notebook? thanks so much!

nagadomi / waifu2x-caffe-ubuntu

Linking error when building on Google Collaboratory #3