[Bug]: Dependency resolution is happening when installing latest version of apache-beam[gcp]

[tvalentyn]

What happened?

Installing apache-beam into a clean environment now shows following logs:

 pip install --no-cache-dir --pre apache-beam[gcp]==2.56.0
...

Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam[gcp]==2.56.0)
  Downloading grpcio_status-1.64.1-py3-none-any.whl.metadata (1.1 kB)
  Downloading grpcio_status-1.64.0-py3-none-any.whl.metadata (1.1 kB)
  Downloading grpcio_status-1.64.0rc1-py3-none-any.whl.metadata (1.1 kB)
  Downloading grpcio_status-1.63.0-py3-none-any.whl.metadata (1.1 kB)
  Downloading grpcio_status-1.63.0rc2-py3-none-any.whl.metadata (1.1 kB)
  Downloading grpcio_status-1.63.0rc1-py3-none-any.whl.metadata (1.1 kB)

That means, when analyzing dependency ranges, pip cannot take the latest allowed version for each dependency, and has to do dependency resolution. This is suboptimal and can result in failed installations (https://s.apache.org/beam-python-dependencies-pm). We should check what combination of packages requires the resolution and work with maintainers of Beam dependencies to remove the need for resolution if possible.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• [X] Component: Python SDK
• [ ] Component: Java SDK
• [ ] Component: Go SDK
• [ ] Component: Typescript SDK
• [ ] Component: IO connector
• [ ] Component: Beam YAML
• [ ] Component: Beam examples
• [ ] Component: Beam playground
• [ ] Component: Beam katas
• [ ] Component: Website
• [ ] Component: Spark Runner
• [ ] Component: Flink Runner
• [ ] Component: Samza Runner
• [ ] Component: Twister2 Runner
• [ ] Component: Hazelcast Jet Runner
• [ ] Component: Google Cloud Dataflow Runner

[tvalentyn]

* grpcio-status==1.66.1
 - protobuf [required: >=5.26.1,<6.0dev, installed: 4.25.5]

Should be fixed by switching to protobuf >=5 or mitigated by adding a bound "grpcio-status<1.63.0"