strongdm / strongdm-sdk-python

strongDM SDK for the Python programming language.
https://strongdm.github.io/strongdm-sdk-python-docs/
Apache License 2.0
9 stars 8 forks source link

fix(protobuf): work around upstream protobuf change #11

Closed hakamadare closed 1 year ago

hakamadare commented 2 years ago

Fixes #10

So, what's all this then? Google released a new major version of Protocol Buffers, and corresponding new protobuf package for Python. See https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates for details.

Since strongdm doesn't constrain the maximum version of protobuf, and neither do any of its upstream dependencies(currently), the following occurs when protobuf is allowed to float above v4:

============================= test session starts ==============================
platform linux -- Python 3.8.12, pytest-7.1.2, pluggy-1.0.0 -- /home/circleci/circleci/strongdm_datasource_generator_venv/bin/python
cachedir: .pytest_cache
rootdir: /home/circleci/circleci
plugins: socket-0.5.1, mock-3.7.0, cov-3.0.0
collecting ... collecting 2 items / 1 error                                                   collected 28 items / 2 errors

==================================== ERRORS ====================================
____ ERROR collecting tools/strongdm_datasource_generator/tests/test_cli.py ____
...
tools/strongdm_datasource_generator/strongdm_utils.py:7: in <module>
    import strongdm
strongdm_datasource_generator_venv/lib/python3.8/site-packages/strongdm/__init__.py:21: in <module>
    from .client import *
strongdm_datasource_generator_venv/lib/python3.8/site-packages/strongdm/client.py:25: in <module>
    from . import svc
strongdm_datasource_generator_venv/lib/python3.8/site-packages/strongdm/svc.py:19: in <module>
    from . import plumbing
strongdm_datasource_generator_venv/lib/python3.8/site-packages/strongdm/plumbing.py:21: in <module>
    from google.rpc import status_pb2
strongdm_datasource_generator_venv/lib/python3.8/site-packages/google/rpc/status_pb2.py:50: in <module>
    _descriptor.FieldDescriptor(
strongdm_datasource_generator_venv/lib/python3.8/site-packages/google/protobuf/descriptor.py:560: in __new__
    _message.Message._CheckCalledFromGeneratedFile()
E   TypeError: Descriptors cannot not be created directly.
E   If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
E   If you cannot immediately regenerate your protos, some other possible workarounds are:
E    1. Downgrade the protobuf package to 3.20.x or lower.
E    2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
E
E   More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
...

:sad-trombone:

However, a few days ago, a new point release of googleapis-common-protos emerged, which includes a dependency constraint on protobuf<4. I suspect a real fix to this problem will require a new release of this module with the protobuf files properly rebuilt; however, as an immediate fix, please consider merging this PR and then promptly publishing a new release of strongdm so that we can pin to strongdm>2.4.0,<3 or some such.

hakamadare commented 2 years ago

@perezjc both i and @andrewmcloud are members of the organization Indigo Agriculture.

JPLachance commented 2 years ago

Hello!

This issue affects Coveo as well. It broke multiple projects that depend on this SDK.

Can we merge the fix and release a new version of the strongDM SDK?

Thank you!!

andrewmcloud commented 2 years ago

@perezjc any updates on this issue?

perezjc commented 2 years ago

Hey @andrewmcloud, happy Friday! A ticket has been opened for this and all updates will be communicated to you by your CSM, Corey. The ticket number is 5466

200sc commented 1 year ago

Changes to these dependency requirements have been in place since v3.4.0; please reopen with an issue if problems are still being encountered.

(A PR is also OK as a suggestion of what change to make, but note that as this repository is generated code, we will be unable to merge the PR.)