runllm[bot]
commented
3 months ago The issue you described involves adding support for protobuf version 5 in Flytekit. This is necessary due to the recent release of protobuf 5.0.0 and its impact on the Flytekit ecosystem, particularly when using the Python backend. The problem is exacerbated by the googleapis-common-protos library dropping constraints on protobuf versions, leading to compatibility issues.
To address this, Flytekit needs to update its dependencies and ensure compatibility with protobuf 5.0.0. This involves modifying the environment variable settings and potentially updating CI plugins that rely on the older protobuf versions.
Would you like more detail on how to proceed with these updates?
Sources:
This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.
Mention @runllm to ask follow-up questions. See runllm.com for more!
eapolinario
Describe the issue
Google released a new major version of the protobuf library in March/2024: https://pypi.org/project/protobuf/#history, but that version started to affect the flytekit ecosystem only after
googleapis-common-protosreleased version 1.36.1, which dropped the constraint on the 5.X version ofprotobuf.Using the new major version of protobuf broke the suggested way of using python backend described in https://github.com/protocolbuffers/protobuf/blob/main/python/README.md#implementation-backends, which involved setting an environment variable to instruct the protobuf runtime to use the python implementation. I opened https://github.com/googleapis/python-api-common-protos/issues/232 to track on their side.
What if we do not do this?
We use some plugins in our CI checks that rely on the python implementation to run, since those haven't updated to the version 4 of the protobuf library.
Related component(s)
flytekit
Are you sure this issue hasn't been raised already?
Have you read the Code of Conduct?