missing dependency `protobuf` for caffe2

peterjc123 commented 4 years ago

🐛 Bug

$> python -c "from caffe2.python import core" 
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\circleci\project\conda\lib\site-packages\caffe2\python\__init__.py", line 2, in <module>
    from caffe2.proto import caffe2_pb2
  File "C:\Users\circleci\project\conda\lib\site-packages\caffe2\proto\__init__.py", line 11, in <module>
    from caffe2.proto import caffe2_pb2, metanet_pb2, torch_pb2
  File "C:\Users\circleci\project\conda\lib\site-packages\caffe2\proto\caffe2_pb2.py", line 5, in <module>
    from google.protobuf.internal import enum_type_wrapper
ModuleNotFoundError: No module named 'google'

To Reproduce

Steps to reproduce the behavior:

conda install -c pytorch-nightly pytorch torchvision / pip install --pre torch torchvision -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html
python -c "from caffe2.python import core"

Expected behavior

Import success.

Environment

Please copy and paste the output from our environment collection script (or fill out the checklist below manually).

You can get the script and run it with:

wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py

PyTorch Version (e.g., 1.0): master
OS (e.g., Linux): Windows
How you installed PyTorch (conda, pip, source): conda/pip
Build command you used (if compiling from source):
Python version: 3.6
CUDA/cuDNN version: N/A
GPU models and configuration: N/A
Any other relevant information:

Additional context

@ezyang @seemethere

cc @malfet

malfet commented 4 years ago

Updated the title, since it's not specific to caffe2.

peterjc123 commented 4 years ago

Updated the title, since it's not specific to caffe2.

You mean Windows, right?

peterjc123 commented 4 years ago

There are two plans:

Add the dependencies to requirements.txt and meta.yml.
Deprecate the python interface for caffe2.python.

What do you think, @ezyang @seemethere @malfet @soumith ?

ezyang commented 4 years ago

@malfet was recently talking about how, for our binary builds, we should just not ship caffe2 by default (you have to build from source if you want caffe2). I think that definitely pushes us towards (2) here. I definitely don't want to add protobuf to the dependency list here.

peterjc123 commented 4 years ago

we should just not ship caffe2 by default

So is it possible to stop compiling them through some CMake options? If we don't want to ship them, then it's just a waste of time building them (at least in release builds).

malfet commented 4 years ago

When it's no longer shipped by default, I think we should still compile caffe2(to avoid regressions), but perhaps for just one configuration on the platform.

peterjc123 commented 4 years ago

@malfet What about still compiling them in PyTorch CI jobs but skipping them in binary jobs?

malfet commented 4 years ago

@peterjc123 sounds reasonable to me.

peterjc123 commented 4 years ago

@malfet So the problem is how to do that using CMake?

peterjc123 commented 4 years ago

Currently I could not find a CMake option to turn off the building of Caffe2 sources.

pytorch / pytorch