TypeError: __init__() got an unexpected keyword argument 'file'

[mihaifieraru]

I get this error

from caffe2.python import core
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/BS/mihai/work/src2/miniconda2/envs/oldgpu_cuda8/lib/python2.7/site-packages/caffe2/python/core.py", line 29, in <module>
    from caffe2.proto import caffe2_pb2
  File "/BS/mihai/work/src2/miniconda2/envs/oldgpu_cuda8/lib/python2.7/site-packages/caffe2/proto/caffe2_pb2.py", line 153, in <module>
    options=None, file=DESCRIPTOR),
TypeError: __init__() got an unexpected keyword argument 'file'

System information

• Operating system: Linux 4.9.84.1.amd64-smp #1 SMP Mon Feb 26 14:10:39 CET 2018 x86_64 GNU/Linux

• Compiler version: gcc version 4.9.2 (Debian 4.9.2-10+deb8u1)

• CMake version: cmake version 3.10.3

• CMake arguments:

  CMAKE_ARGS+=("-DCUDA_TOOLKIT_ROOT_DIR=/BS/mihai/work/src2/cuda-8.0/")
  CMAKE_ARGS+=("-DNCCL_INCLUDE_DIRS=/BS/mihai/work/src2/nccl_2.1.4-1+cuda8.0_x86_64/include/")
  CMAKE_ARGS+=("-DNCCL_LIBRARIES=/BS/mihai/work/src2/nccl_2.1.4-1+cuda8.0_x86_64/lib/")

• Relevant libraries/versions (e.g. CUDA): cuda-8.0 , cudnn 7.1.1 , nccl_2.1.4

CMake summary output

-- ******** Summary ********
-- General:
--   CMake version         : 3.10.3
--   CMake command         : /BS/mihai/work/src2/miniconda2/conda-bld/caffe2-cuda-cudnn-gcc4.9_1521670563651/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/bin/cmake
--   Git version           : v0.8.1-1390-g6283229-dirty
--   System                : Linux
--   C++ compiler          : /usr/bin/c++
--   C++ compiler version  : 4.9.2
--   Protobuf compiler     : /BS/mihai/work/src2/miniconda2/conda-bld/caffe2-cuda-cudnn-gcc4.9_1521670563651/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/bin/protoc
--   Protobuf include path : /BS/mihai/work/src2/miniconda2/conda-bld/caffe2-cuda-cudnn-gcc4.9_1521670563651/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/include
--   Protobuf libraries    : /BS/mihai/work/src2/miniconda2/conda-bld/caffe2-cuda-cudnn-gcc4.9_1521670563651/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/libprotobuf.so;-lpthread
--   BLAS                  : Eigen
--   CXX flags             : -D_GLIBCXX_USE_CXX11_ABI=0 -Wno-deprecated -DONNX_NAMESPACE=onnx_c2 -O2 -fPIC -Wno-narrowing -Wno-invalid-partial-specialization
--   Build type            : Release
--   Compile definitions   : 
-- 
--   BUILD_BINARY          : ON
--   BUILD_CUSTOM_PROTOBUF : OFF
--   BUILD_DOCS            : OFF
--   BUILD_PYTHON          : ON
--     Python version      : 2.7.14
--     Python includes     : /BS/mihai/work/src2/miniconda2/conda-bld/caffe2-cuda-cudnn-gcc4.9_1521670563651/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/include/python2.7
--   BUILD_SHARED_LIBS     : ON
--   BUILD_TEST            : ON
--   USE_ATEN              : OFF
--   USE_ASAN              : OFF
--   USE_CUDA              : ON
--     CUDA version        : 8.0
--     CuDNN version       : 7.1.1
--     CUDA root directory : /BS/mihai/work/src2/cuda-8.0/
--     CUDA library        : /usr/lib/x86_64-linux-gnu/libcuda.so
--     CUDA NVRTC library  : /usr/lib/x86_64-linux-gnu/libnvrtc.so
--     CUDA runtime library: /BS/mihai/work/src2/cuda-8.0/lib64/libcudart.so
--     CUDA include path   : /BS/mihai/work/src2/cuda-8.0/include
--     NVCC executable     : /BS/mihai/work/src2/cuda-8.0/bin/nvcc
--     CUDA host compiler  : /usr/bin/cc
--   USE_EIGEN_FOR_BLAS    : 1
--   USE_FFMPEG            : OFF
--   USE_GFLAGS            : ON
--   USE_GLOG              : ON
--   USE_GLOO              : ON
--   USE_LEVELDB           : ON
--     LevelDB version     : 1.18
--     Snappy version      : ..
--   USE_LITE_PROTO        : OFF
--   USE_LMDB              : ON
--     LMDB version        : 0.9.21
--   USE_METAL             : OFF
--   USE_MKL               : 
--   USE_MOBILE_OPENGL     : OFF
--   USE_MPI               : OFF
--   USE_NCCL              : ON
--   USE_NERVANA_GPU       : OFF
--   USE_NNPACK            : ON
--   USE_OBSERVERS         : ON
--   USE_OPENCV            : ON
--     OpenCV version      : 3.1.0
--   USE_OPENMP            : OFF
--   USE_PROF              : OFF
--   USE_REDIS             : OFF
--   USE_ROCKSDB           : OFF
--   USE_ZMQ               : OFF

I install caffe2 via:

CONDA_INSTALL_LOCALLY=1 BUILD_ENVIRONMENT=-cuda- scripts/build_anaconda.sh

This is some selective output with the packages that are installed:

## Package Plan ##

  environment location: /BS/mihai/work/src2/miniconda2/conda-bld/caffe2-cuda-cudnn-gcc4.9_1521670563651/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol

The following NEW packages will be INSTALLED:

    blas:            1.1-openblas                  conda-forge
    bzip2:           1.0.6-1                       conda-forge
    ca-certificates: 2018.1.18-0                   conda-forge
    certifi:         2018.1.18-py27_0              conda-forge
    cmake:           3.10.3-0                      conda-forge
    curl:            7.59.0-0                      conda-forge
    expat:           2.2.5-0                       conda-forge
    future:          0.16.0-py27_0                 conda-forge
    gflags:          2.2.1-0                       conda-forge
    glog:            0.3.5-0                       conda-forge
    hdf5:            1.8.17-11                     conda-forge
    jasper:          1.900.1-4                     conda-forge
    jpeg:            9b-2                          conda-forge
    krb5:            1.14.2-0                      conda-forge
    leveldb:         1.18-0                        conda-forge
    libgfortran:     3.0.0-1                                  
    libpng:          1.6.34-0                      conda-forge
    libprotobuf:     3.5.1.1-0                     conda-forge
    libssh2:         1.8.0-2                       conda-forge
    libtiff:         4.0.9-0                       conda-forge
    libuv:           1.19.2-0                      conda-forge
    lmdb:            0.9.21-0                      conda-forge
    ncurses:         5.9-10                        conda-forge
    numpy:           1.11.3-py27_blas_openblas_203 conda-forge [blas_openblas]
    openblas:        0.2.20-7                      conda-forge
    opencv:          3.1.0-np111py27_1             conda-forge
    openssl:         1.0.2n-0                      conda-forge
    pip:             9.0.1-py27_1                  conda-forge
    protobuf:        3.5.1.1-py27_0                conda-forge
    python:          2.7.14-4                      conda-forge
    readline:        7.0-0                         conda-forge
    rhash:           1.3.4-0                       conda-forge
    setuptools:      39.0.1-py27_0                 conda-forge
    six:             1.11.0-py27_1                 conda-forge
    snappy:          1.1.7-1                       conda-forge
    sqlite:          3.20.1-2                      conda-forge
    tk:              8.6.7-0                       conda-forge
    wheel:           0.30.0-py27_2                 conda-forge
    xz:              5.2.3-0                       conda-forge
    zlib:            1.2.11-0                      conda-forge

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done

# and after building, I get:

Solving environment: ...working... done

## Package Plan ##

  environment location: /BS/mihai/work/src2/miniconda2/conda-bld/caffe2-cuda-cudnn-gcc4.9_1521670563651/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place

The following NEW packages will be INSTALLED:

    blas:                     1.1-openblas                  conda-forge
    ca-certificates:          2018.1.18-0                   conda-forge
    caffe2-cuda-cudnn-gcc4.9: 0.8.dev-py27hc501f1c_0        local      
    certifi:                  2018.1.18-py27_0              conda-forge
    future:                   0.16.0-py27_0                 conda-forge
    gflags:                   2.2.1-0                       conda-forge
    glog:                     0.3.5-0                       conda-forge
    hdf5:                     1.8.17-11                     conda-forge
    jasper:                   1.900.1-4                     conda-forge
    jpeg:                     9b-2                          conda-forge
    leveldb:                  1.18-0                        conda-forge
    libgfortran:              3.0.0-1                                  
    libpng:                   1.6.34-0                      conda-forge
    libprotobuf:              3.5.2-0                       conda-forge
    libtiff:                  4.0.9-0                       conda-forge
    lmdb:                     0.9.21-0                      conda-forge
    ncurses:                  5.9-10                        conda-forge
    numpy:                    1.12.1-py27_blas_openblas_201 conda-forge [blas_openblas]
    openblas:                 0.2.20-7                      conda-forge
    opencv:                   3.1.0-np112py27_1             conda-forge
    openssl:                  1.0.2n-0                      conda-forge
    pip:                      9.0.1-py27_1                  conda-forge
    protobuf:                 3.5.2-py27_0                  conda-forge
    python:                   2.7.14-4                      conda-forge
    readline:                 7.0-0                         conda-forge
    setuptools:               39.0.1-py27_0                 conda-forge
    six:                      1.11.0-py27_1                 conda-forge
    snappy:                   1.1.7-1                       conda-forge
    sqlite:                   3.20.1-2                      conda-forge
    tk:                       8.6.7-0                       conda-forge
    wheel:                    0.30.0-py27_2                 conda-forge
    xz:                       5.2.3-0                       conda-forge
    zlib:                     1.2.11-0                      conda-forge

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
import: u'caffe2.python.core'
import: u'caffe2.python.core'
import: u'caffe2.python.core'

Resource usage statistics from testing caffe2-cuda-cudnn-gcc4.9:
   Process count: 2
   CPU time: Sys=0:00:01.6, User=0:00:00.9
   Memory: 37.4M
   Disk usage: 3B
   Time elapsed: 0:00:04.2

TEST END: /BS/mihai/work/src2/miniconda2/conda-bld/linux-64/caffe2-cuda-cudnn-gcc4.9-0.8.dev-py27hc501f1c_0.tar.bz2
# Automatic uploading is disabled
# If you want to upload package(s) to anaconda.org later, type:

anaconda upload /BS/mihai/work/src2/miniconda2/conda-bld/linux-64/caffe2-cuda-cudnn-gcc4.9-0.8.dev-py27hc501f1c_0.tar.bz2

# To have conda build upload to anaconda.org automatically, use
# $ conda config --set anaconda_upload yes

anaconda_upload is not set.  Not uploading wheels: []
####################################################################################
Resource usage summary:

Total time: 0:19:43.2
CPU usage: sys=0:04:07.4, user=0:47:16.1
Maximum memory usage observed: 9.5G
Total disk usage observed (not including envs): 931.9M

####################################################################################
Source and build intermediates have been left in /BS/mihai/work/src2/miniconda2/conda-bld.
There are currently 1 accumulated.
To remove them, you can run the ```conda build purge``` command
+ '[' -n 1 ']'
+ conda install -y -c conda-forge caffe2-cuda-cudnn-gcc4.9 --use-local
Solving environment: done

## Package Plan ##

  environment location: /BS/mihai/work/src2/miniconda2/envs/oldgpu_cuda8

  added / updated specs: 
    - caffe2-cuda-cudnn-gcc4.9

The following NEW packages will be INSTALLED:

    caffe2-cuda-cudnn-gcc4.9: 0.8.dev-py27hc501f1c_0 local      
    future:                   0.16.0-py27_0          conda-forge
    gflags:                   2.2.1-0                conda-forge
    glog:                     0.3.5-0                conda-forge
    hdf5:                     1.8.17-11              conda-forge
    intel-openmp:             2018.0.0-hc7b2577_8               
    jasper:                   1.900.1-4              conda-forge
    jpeg:                     9b-2                   conda-forge
    leveldb:                  1.18-0                 conda-forge
    libgfortran:              3.0.0-1                           
    libgfortran-ng:           7.2.0-hdf63c60_3                  
    libpng:                   1.6.34-0               conda-forge
    libprotobuf:              3.5.2-0                conda-forge
    libtiff:                  4.0.9-0                conda-forge
    lmdb:                     0.9.21-0               conda-forge
    mkl:                      2018.0.1-h19d6760_4               
    numpy:                    1.12.1-py27h9378851_1             
    opencv:                   3.1.0-np112py27_1      conda-forge
    protobuf:                 3.5.2-py27_0           conda-forge
    six:                      1.11.0-py27_1          conda-forge
    snappy:                   1.1.7-1                conda-forge
    xz:                       5.2.3-0                conda-forge

The following packages will be UPDATED:

    ca-certificates:          2017.08.26-h1d4fec5_0              --> 2018.1.18-0      conda-forge
    certifi:                  2018.1.18-py27_0                   --> 2018.1.18-py27_0 conda-forge
    openssl:                  1.0.2n-hb7f436b_0                  --> 1.0.2n-0         conda-forge

Preparing transaction: done
Verifying transaction: done
Executing transaction: done

Is it not very weird that conda build uses

    libprotobuf:     3.5.1.1-0                     conda-forge

yet, after build, when the package is installed, this depenency is installed:

    libprotobuf:              3.5.2-0                conda-forge

?

[pjh5]

@fierarufmihai can you manually install the same protobuf used during installation? I think the command is conda install -y protobuf=3.5.1.1=py27_0 -c conda-forge