File Limit Request: pymarian - 650MB

thammegowda commented 3 months ago

Project URL

https://pypi.org/project/pymarian/

Does this project already exist?

[X] Yes

New Limit

650MB

Update issue title

[X] I have updated the title.

Which indexes

PyPI

About the project

Pymarian offers python binding to Marian NMT.

Marian NMT is a fast neural machine translation toolkit widely used across industry (e.g. Microsoft) and academic researchers.
Pymarian project brings the best of both worlds -- the speed of c++ code and the convenience of Python API. Our library allows fast machine translation, language modeling (e.g. transformers), and evaluation (such as COMET and BLEURT scores).

We have not included any data or examples. We create native extension using Pybind11. To increase portability of code. we static link certain libraries such as Intel MKL, and Nvidia CUDA. Since these libraries are huge, we are seeing increased file size for our native extension.

Reasons for the request

Without this limit increase, users of our project have to compile Marian C++ code along with CUDA libraries, which can be complicated for some users than simply running "pip install pymarian" to get started. Our project uses Nvidia CUDA libraries. For maximum portability, we statically link to cuda libraries. This increases our native extension size, and the wheel file size to 600+ MB. Our scenario is similar to pytorch, e.g. https://pypi.org/project/torch/#files where their wheel files (with cuda support) are~800MB. Kindly increase the limit to 650MB to facilitate release of our project.

Code of Conduct

[X] I agree to follow the PSF Code of Conduct

alvations commented 3 months ago

@di might be able to help.

cmaureir commented 3 months ago

Hello @thammegowda :wave: I've set the upload limit for pymarian to 650 MB on PyPI. Please be mindful of the frequency of releases at that size. Have a nice week :rocket:

thammegowda commented 3 months ago

Thank you @cmaureir !

pypi / support