search

uber / neuropod

A uniform interface to run deep learning models from multiple frameworks
https://neuropod.ai
Apache License 2.0
936 stars 77 forks source link

[Python] Pin protobuf version to fix tensorflow #546

Closed VivekPanyam closed 2 years ago

VivekPanyam commented 2 years ago

Summary:

A major release of the protobuf python package (4.21.0) came out a few hours ago. It appears that tensorflow doesn't restrict the allowed versions of its protobuf dependency. This causes TF 2.2.0 to break and therefore breaks Neuropod CI.

An excerpt of the error:

TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

More details in the comments of this PR

Test Plan:

CI

VivekPanyam commented 2 years ago

Some of the failing tests caused by this issue:

======================================================================
ERROR: backends.keras.test.test_keras_packaging (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: backends.keras.test.test_keras_packaging
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/Users/runner/work/neuropod/neuropod/source/python/neuropod/backends/keras/test/test_keras_packaging.py", line 16, in <module>
    import tensorflow as tf
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/__init__.py", line 41, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/python/__init__.py", line 53, in <module>
    from tensorflow.core.framework.graph_pb2 import *
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/graph_pb2.py", line 16, in <module>
    from tensorflow.core.framework import function_pb2 as tensorflow_dot_core_dot_framework_dot_function__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/function_pb2.py", line 16, in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py", line 16, in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py", line 36, in <module>
    _descriptor.FieldDescriptor(
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/google/protobuf/descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

======================================================================
ERROR: backends.tensorflow.test.custom_ops.test_tensorflow_custom_ops (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: backends.tensorflow.test.custom_ops.test_tensorflow_custom_ops
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/Users/runner/work/neuropod/neuropod/source/python/neuropod/backends/tensorflow/test/custom_ops/test_tensorflow_custom_ops.py", line 19, in <module>
    import tensorflow as tf
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/__init__.py", line 41, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/python/__init__.py", line 53, in <module>
    from tensorflow.core.framework.graph_pb2 import *
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/graph_pb2.py", line 16, in <module>
    from tensorflow.core.framework import function_pb2 as tensorflow_dot_core_dot_framework_dot_function__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/function_pb2.py", line 16, in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py", line 16, in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py", line 36, in <module>
    _descriptor.FieldDescriptor(
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/google/protobuf/descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

======================================================================
ERROR: backends.tensorflow.test.test_tensorflow_packaging (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: backends.tensorflow.test.test_tensorflow_packaging
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/Users/runner/work/neuropod/neuropod/source/python/neuropod/backends/tensorflow/test/test_tensorflow_packaging.py", line 17, in <module>
    import tensorflow as tf
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/__init__.py", line 41, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/python/__init__.py", line 53, in <module>
    from tensorflow.core.framework.graph_pb2 import *
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/graph_pb2.py", line 16, in <module>
    from tensorflow.core.framework import function_pb2 as tensorflow_dot_core_dot_framework_dot_function__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/function_pb2.py", line 16, in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py", line 16, in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py", line 36, in <module>
    _descriptor.FieldDescriptor(
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/google/protobuf/descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

======================================================================
ERROR: backends.tensorflow.test.test_tensorflow_strings (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: backends.tensorflow.test.test_tensorflow_strings
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/Users/runner/work/neuropod/neuropod/source/python/neuropod/backends/tensorflow/test/test_tensorflow_strings.py", line 16, in <module>
    import tensorflow as tf
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/__init__.py", line 41, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/python/__init__.py", line 53, in <module>
    from tensorflow.core.framework.graph_pb2 import *
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/graph_pb2.py", line 16, in <module>
    from tensorflow.core.framework import function_pb2 as tensorflow_dot_core_dot_framework_dot_function__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/function_pb2.py", line 16, in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py", line 16, in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py", line 36, in <module>
    _descriptor.FieldDescriptor(
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/google/protobuf/descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

======================================================================
ERROR: backends.tensorflow.test.test_tensorflow_v2_packaging (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: backends.tensorflow.test.test_tensorflow_v2_packaging
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/Users/runner/work/neuropod/neuropod/source/python/neuropod/backends/tensorflow/test/test_tensorflow_v2_packaging.py", line 16, in <module>
    import tensorflow as tf
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/__init__.py", line 41, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/python/__init__.py", line 53, in <module>
    from tensorflow.core.framework.graph_pb2 import *
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/graph_pb2.py", line 16, in <module>
    from tensorflow.core.framework import function_pb2 as tensorflow_dot_core_dot_framework_dot_function__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/function_pb2.py", line 16, in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py", line 16, in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py", line 36, in <module>
    _descriptor.FieldDescriptor(
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/google/protobuf/descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

======================================================================
ERROR: utils.test.test_randomify (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: utils.test.test_randomify
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/Users/runner/work/neuropod/neuropod/source/python/neuropod/utils/test/test_randomify.py", line 20, in <module>
    import tensorflow as tf
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/__init__.py", line 41, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/python/__init__.py", line 53, in <module>
    from tensorflow.core.framework.graph_pb2 import *
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/graph_pb2.py", line 16, in <module>
    from tensorflow.core.framework import function_pb2 as tensorflow_dot_core_dot_framework_dot_function__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/function_pb2.py", line 16, in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py", line 16, in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py", line 36, in <module>
    _descriptor.FieldDescriptor(
  File "/Users/runner/work/neuropod/neuropod/.neuropod_venv/lib/python3.8/site-packages/google/protobuf/descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
codecov[bot] commented 2 years ago

Codecov Report

Merging #546 (585263f) into master (c79f8af) will increase coverage by 0.05%. The diff coverage is 93.87%.

:exclamation: Current head 585263f differs from pull request most recent head 5a826d1. Consider uploading reports for the commit 5a826d1 to get more accurate results

@@            Coverage Diff             @@
##           master     #546      +/-   ##
==========================================
+ Coverage   86.58%   86.63%   +0.05%     
==========================================
  Files         105      105              
  Lines        6975     7002      +27     
==========================================
+ Hits         6039     6066      +27     
  Misses        936      936
Impacted Files Coverage Δ
...hon_bridge/_neuropod_native_bootstrap/pip_utils.py 65.51% <50.00%> (ø)
...e/neuropod/backends/python_bridge/python_bridge.cc 83.24% <100.00%> (ø)
...rce/neuropod/backends/torchscript/torch_backend.cc 81.26% <100.00%> (+1.43%) :arrow_up:
source/python/neuropod/backends/python/utils.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 37fc1f4...5a826d1. Read the comment docs.