rioyokotalab / caffe2

Caffe2 is a lightweight, modular, and scalable deep learning framework.
https://caffe2.ai
Other
2 stars 0 forks source link

[Redis Set up Error] #8

Closed Hiroki11x closed 7 years ago

Hiroki11x commented 7 years ago

I installed Redis to /path-to-redis/ as following instaruction https://github.com/kurosawatsuyoshi/doshelper/wiki/1.-redis-Setup%EF%BC%88redis%E3%81%AE%E3%82%BB%E3%83%83%E3%83%88%E3%82%A2%E3%83%83%E3%83%97%EF%BC%89

redis-server
[80640] 17 Jul 22:37:55.764 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
[80640] 17 Jul 22:37:55.765 * Increased maximum number of open files to 10032 (it was originally set to 4096).
[80640] 17 Jul 22:37:55.765 # Creating Server TCP listening socket *:6379: bind: Address already in use

https://redis.io/topics/quickstart

then, don't mind

I installed Hiredis(This is redis connection library.)

$ wget -O hiredis.zip https://github.com/redis/hiredis/archive/master.zip
$ unzip hiredis.zip
$ cd hiredis-master/

you should edit Makefile as following

 17 # Installation related variables and target
 18 PREFIX=/path-to/local/hiredis
 19 INCLUDE_PATH=include/hiredis
 20 LIBRARY_PATH=lib

then execute

$ make
$ sudo make install

after that, rebuild caffe2

CMAKE_PREFIX_PATH=/path-to/opencv-2.4.13:/path-to/snappy_1.1.4:/path-to/redis-2.8.12 cmake .. \
-DBLAS=Eigen \
-DUSE_CUDA=ON \
-DUSE_ROCKSDB=OFF \
-DUSE_GLOO=ON \
-DUSE_REDIS=ON \
-DUSE_OPENCV=ON \
-DUSE_GFLAGS=OFF \
-DCUDNN_INCLUDE_DIR=/path-to/cuda/include \
-DCUDNN_LIBRARY=/path-to/cuda/lib/libcudnn.so \
-DCMAKE_INSTALL_PREFIX=/path-to/caffe2/local \
-DMPI_C_COMPILER=/path-to/openmpi-2.0.1/xl/bin/mpicc \
-DMPI_CXX_COMPILER=/path-to/openmpi-2.0.1/xl/bin/mpicxx

console out put

-- ******** Summary ********
-- General:
--   Git version           : 
--   System                : Linux
--   C++ compiler          : /usr/bin/c++
--   C++ compiler version  : 4.8.5
--   Protobuf compiler     : /usr/bin/protoc
--   CXX flags             :  -fopenmp -std=c++11 -fPIC -Wno-narrowing
--   Build type            : Release
--   Compile definitions   : CAFFE2_USE_EIGEN_FOR_BLAS;CAFFE2_USE_GOOGLE_GLOG;EIGEN_MPL2_ONLY;CAFFE2_FORCE_FALLBACK_CUDA_MPI;CAFFE2_NO_BUILTIN_CPU_SUPPORTS
-- 
--   BUILD_SHARED_LIBS     : ON
--   BUILD_PYTHON          : ON
--     Python version      : 2.7.5
--     Python library      : /usr/lib64/libpython2.7.so
--   BUILD_TEST            : ON
--   USE_CUDA              : ON
--     CUDA version        : 8.0
--   USE_CNMEM             : OFF
--   USE_NERVANA_GPU       : OFF
--   USE_GLOG              : ON
--   USE_GFLAGS            : OFF
--   USE_LMDB              : ON
--     LMDB version        : 0.9.18
--   USE_LEVELDB           : ON
--     LevelDB version     : 1.20
--     Snappy version      : 1.1.4
--   USE_OPENCV            : ON
--     OpenCV version      : 2.4.13
--   USE_FFMPEG            : 
--   USE_ZMQ               : OFF
--   USE_ROCKSDB           : OFF
--   USE_MPI               : ON
--   USE_NCCL              : ON
--   USE_NNPACK            : OFF
--   USE_OPENMP            : ON
--   USE_REDIS             : ON
--   USE_GLOO              : ON
-- Configuring done
-- Generating done
Hiroki11x commented 7 years ago

https://stackoverflow.com/questions/36880321/why-redis-can-not-set-maximum-open-file

Hiroki11x commented 7 years ago
# Creating Server TCP listening socket *:6379: bind: Address already in use

Apparently other servers seem to be working http://qiita.com/horiko/items/bc812a03c9e0566d6338

confirm that,following code

$ redis-cli ping
PONG
Hiroki11x commented 7 years ago

How to use redis-cli http://qiita.com/sawada_masahiko/items/1f60936c421ecab8dfbf

Hiroki11x commented 7 years ago

http://kenken0807.hatenablog.com/entry/2016/06/10/103656

Protected mode is valid after redis 3.2.0

https://redis.io/

wget http://download.redis.io/releases/redis-4.0.0.tar.gz
tar zxvf redis-4.0.0.tar.gz
cd redis-4.0.0
make -j 128
make install PREFIX=/path-to-install
Hiroki11x commented 7 years ago

when execute distributed training, the following error message appeared

ValueError: Unknown argument type: key=prefix value=None, value type=<type 'NoneType'

https://github.com/caffe2/caffe2/issues/482

So, I reinstall protobuf & rebuild Caffe2 by adding bellow CMAKE_PREFIX_PATH

CMAKE_PREFIX_PATH=/path-to/protobuf-3.2.0