search

pypi / warehouse

The Python Package Index
https://pypi.org
Apache License 2.0
3.6k stars 963 forks source link

Request for file size limit exemption #5542

Closed vthorsteinsson closed 5 years ago

vthorsteinsson commented 5 years ago

Dear PyPI, I hereby request an exemption from the file size limit for PyPI (not TestPyPI) for my project Icegrams (https://pypi.org/project/icegrams/). Icegrams is a Python package that encapsulates a heavily compressed database of trigrams for Icelandic text. The database contains 34 million trigrams that occupy 788 megabytes in uncompressed text format. Using several tricks of the trade, including succinct Elias-Fano indexes and radix tries, Icegrams squeezes this data plus the associated Python code into an 80.4 megabyte downloadable package, which exceeds PyPI's limit. Icegrams is a very useful tool for Icelandic Natural Language Processing and being able to install it in one simple step via pip helps its usability and adoption. I do not anticipate that the database will become bigger than this over time. I hope this request meets with your understanding.

jamadden commented 5 years ago

Thanks for the detailed report. I've increased the size limit to 100MB on pypi.org.