search

pypi / support

Issue tracker for support requests related to using https://pypi.org
90 stars 47 forks source link

Project Limit Request: grpcio - 100 GB #1921

Closed lidizheng closed 2 years ago

lidizheng commented 2 years ago

Project URL

https://pypi.org/project/grpcio/

Does this project already exist?

New limit

100GB

Update issue title

Which indexes

PyPI, TestPyPI

About the project

grpcio package is the Python implementation of the gRPC protocol. gRPC is a modern open source high performance Remote Procedure Call (RPC) framework. It's the #59 popular packages on PyPI based on download counts (https://hugovk.github.io/top-pypi-packages/). gRPC Python wraps gRPC Core as a C extension and we are supporting major platforms and cpu architectures: Linux/macOS/Windows, 86/64/ARMv7, manylinux/musllinux.

The project's first release is around 2015, and we have ~200 releases over the years.

How large is each release?

~850 MB per release

Source Distribution: ~22 MB tarball

Binary Distribution for One Python Version (we support 3.6/3.7/3.8/3.9/3.10):

How frequently do you make a release?

Our release cycles are around every 6 weeks. It contains at least one pre-release and one official release, patch releases or multiple pre-releases might happen occasionally.

Conservatively, let's say our release frequency is 3 releases / 6 weeks.

Code of Conduct

lidizheng commented 2 years ago

CC @gnossen

cmaureir commented 2 years ago

Out of curiosity, have you consider adopting the limited API to reduce the amount of wheels you have per platform?

lidizheng commented 2 years ago

That's interesting! This is the first time I know there is a way to compile C Extensions that can be compatible for multiple Python versions. We are using Cython as the glue layer between C/C++ and Python. Reading the Cython change logs, they are providing support for limited API: https://cython.readthedocs.io/en/latest/src/changes.html. There might be some blockers we need to solve, e.g., asyncio usages, API compatibilities.

This is a good direction to explore, we might not have enough eng resources at this point (call for volunteers!) to act on it now. Our current usage is ~29GB, while the limit is 30GB. It will improve the QoL of our release managers if we can increase the project limit soon.

lidizheng commented 2 years ago

Let me know if anything is blocking this request.

cmaureir commented 2 years ago

I've set the project limit for grpcio to 100 GB on PyPI and TestPyPI. I hope the people involved have the bandwidth to explore the limited API approach, it might be a bit tricky at the beginning, but a real life saver when having so many platforms, and versions.

lidizheng commented 2 years ago

@cmaureir Thank you! We will consider investing the limited API approach in our planning meet up.