Open discordianfish opened 1 year ago
To sum this up:
protobuf (<3.20,>=3.9.2)
$ for v in 2.11.0 2.10.1 2.10.0 2.9.3 2.9.2 2.9.1 2.8.4; do echo "$v: $(curl -s https://pypi.org/pypi/tensorflow/$v/json | jq .info.requires_dist|grep protobuf)"; done
2.11.0: "protobuf (<3.20,>=3.9.2)",
2.10.1: "protobuf (<3.20,>=3.9.2)",
2.10.0: "protobuf (<3.20,>=3.9.2)",
2.9.3: "protobuf (<3.20,>=3.9.2)",
2.9.2: "protobuf (<3.20,>=3.9.2)",
2.9.1: "protobuf (<3.20,>=3.9.2)",
2.8.4: "protobuf (<3.20,>=3.9.2)",
protobuf (<3.20,>=3.9.2)
as well$ for v in 1.51.1 1.50.0 1.49.1 1.48.2; do echo "$v: $(curl -s https://pypi.org/pypi/grpcio-tools/$v/json | jq .info.requires_dist|grep protobuf)"; done
1.51.1: "protobuf (<5.0dev,>=4.21.6)",
1.50.0: "protobuf (<5.0dev,>=4.21.6)",
1.49.1: "protobuf (<5.0dev,>=4.21.3)",
1.48.2: "protobuf (<4.0dev,>=3.12.0)",
Therefor we need to:
protobuf (<3.20,>=3.9.2)
, e.g 1.48.2
protobuf (<4.0dev,>=3.12.0)
in our setup.pyThat will allow users to install ray-rllib where pip dep resolver should install protobuf <3.20,>=3.9.2
while allowing users not depending on ray/tf2 to also install <=3.12.0 and >=3.20.
Currently blocked to 1.48.2 because of TF 2.x limitation of protobuf < 3.20 (and TF is currently used in Ray RL Lib). This also limit python version to 3.11 as it requires a newer grpcio version (above 1.48)
Files affected:
build-pip-package
Other relevant info
Precondition on protobuf changed between: grpcio-tools 1.48.2 https://github.com/grpc/grpc/blob/5e97a6003d0e44bdf3c4234d37b9509d02bf55ea/tools/distrib/python/grpcio_tools/setup.py#L293 and 1.49: https://github.com/grpc/grpc/blob/8f8edfd04b46ee67f90454b3f6a70aa58ff82c2d/tools/distrib/python/grpcio_tools/setup.py#L293
Compatibility problem of protobuf https://stackoverflow.com/questions/71759248/importerror-cannot-import-name-builder-from-google-protobuf-internal