PPQ installtion error: `TypeError: Descriptors cannot be created directly. If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.`

[kyrie2to11]

Hi there,

As a beginner learning about quantization, I encountered an error related to protobuf while installing PPQ. I resolved it by downgrading protobuf using the command pip install protobuf==3.20.*. I'm curious to know if there are any implicit consequences of completing the installation in this way. Here is the detailed error information:

(ppq) jarvis@zephyrus:~/Projects/ppq$ python setup.py install
/home/jarvis/Projects/ppq/ppq/__init__.py:10: UserWarning: You are importing ppq within its own root folder (/home/jarvis/Projects/ppq). 
  warnings.warn(message.format(os.getcwd()))

      ____  ____  __   ____                    __              __
     / __ \/ __ \/ /  / __ \__  ______ _____  / /_____  ____  / /
    / /_/ / /_/ / /  / / / / / / / __ `/ __ \/ __/ __ \/ __ \/ /
   / ____/ ____/ /__/ /_/ / /_/ / /_/ / / / / /_/ /_/ / /_/ / /
  /_/   /_/   /_____\___\_\__,_/\__,_/_/ /_/\__/\____/\____/_/

Traceback (most recent call last):
  File "setup.py", line 2, in <module>
    from ppq.core import PPQ_CONFIG
  File "/home/jarvis/Projects/ppq/ppq/__init__.py", line 13, in <module>
    from ppq.api.setting import (ActivationQuantizationSetting, DispatchingTable,
  File "/home/jarvis/Projects/ppq/ppq/api/__init__.py", line 1, in <module>
    from ppq.lib import (register_calibration_observer, register_network_exporter,
  File "/home/jarvis/Projects/ppq/ppq/lib/__init__.py", line 44, in <module>
    from .extension import (register_calibration_observer,
  File "/home/jarvis/Projects/ppq/ppq/lib/extension.py", line 7, in <module>
    from .common import __EXPORTERS__, __PARSERS__, __QUANTIZER_COLLECTION__
  File "/home/jarvis/Projects/ppq/ppq/lib/common.py", line 2, in <module>
    from ppq.parser import (AscendExporter, CaffeExporter, CaffeParser,
  File "/home/jarvis/Projects/ppq/ppq/parser/__init__.py", line 4, in <module>
    from .caffe_exporter import (CaffeExporter, PPLDSPCaffeExporter,
  File "/home/jarvis/Projects/ppq/ppq/parser/caffe_exporter.py", line 15, in <module>
    from .caffe import ppl_caffe_pb2
  File "/home/jarvis/Projects/ppq/ppq/parser/caffe/ppl_caffe_pb2.py", line 31, in <module>
    _descriptor.EnumValueDescriptor(
  File "/home/jarvis/miniconda3/envs/ppq/lib/python3.8/site-packages/google/protobuf/descriptor.py", line 789, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

I would like to understand if there might be any unintended side effects or results from using the downgrade approach in this situation. Thank you for your assistance!

[ZhangZhiPku]

出现上述问题的原因是PPQ在过去支持过很多公司内部的 caffe 模型，这些 caffe 模型历史久远，它们使用 protobuf3 进行序列化。 因此 PPQ 的 caffe_exporter 需要引用 protobuf3。

解决问题的方法很简单，您注意到报错位于 File "/home/jarvis/Projects/ppq/ppq/parser/caffe_exporter.py", line 15, in from .caffe import ppl_caffe_pb2

请将其直接修改成

try:
    from .caffe import ppl_caffe_pb2
except Exception as e:
    pass

上述修改将直接屏蔽掉 protobuf3 的引用，它只在导出 caffe model 时起作用，在下一个 ppq 的版本中我们也将如此更新。

[zhangcheng030970]

[zhangcheng030970]

修改后任然报错

[zhangcheng030970]

出现上述问题的原因是PPQ在过去支持过很多公司内部的 caffe 模型，这些 caffe 模型历史久远，它们使用 protobuf3 进行序列化。 因此 PPQ 的 caffe_exporter 需要引用 protobuf3。

解决问题的方法很简单，您注意到报错位于 File "/home/jarvis/Projects/ppq/ppq/parser/caffe_exporter.py", line 15, in from .caffe import ppl_caffe_pb2

请将其直接修改成

try:
    from .caffe import ppl_caffe_pb2
except Exception as e:
    pass

上述修改将直接屏蔽掉 protobuf3 的引用，它只在导出 caffe model 时起作用，在下一个 ppq 的版本中我们也将如此更新。

[J-xinyu]

you need a lower vesion of onnx. it works for me pip install onnx==1.13.0