search

pypi / support

Issue tracker for support requests related to using https://pypi.org
95 stars 47 forks source link

Project Limit Request: honeybee-energy - 15 GB #1829

Closed chriswmackey closed 2 years ago

chriswmackey commented 2 years ago

Project URL

https://pypi.org/project/honeybee-energy/

Does this project already exist?

New limit

10 GB? I cannot understand how we hit the limit

Update issue title

Which indexes

PyPI

About the project

We are an open source project that connects Computer Aided Design interfaces to simulation engines that estimate building energy use and model occupant comfort. More information can be found on our website.

Our Python libraries are relatively small but we have subdivided them into different packages as can be seen in our SDK docs here. Given restrictions of how pip resolves dependencies, this has necessitated that we perform a lot of releases to keep the versions of these packages synced and compatible with one another.

Hopefully it goes without saying that PyPI provides a mission-critical service for us and our development would grid to a halt without it. The business we have founded around our open source software is not yet profitable but, if there was any way to purchase additional space for our packages from PyPI, we would undoubtedly do so.

How large is each release?

The project is currently 428.7 kB total (or roughly 0.5 MB): image

We have performed ~500 releases of the project: image

Taking this together, the amount of space consumed by the project should be 0.5 MB * 500 releases = 250 MB. So I was expecting that we were not even at 1 GB yet and I don't know how we have hit the 10 GB limit.

How frequently do you make a release?

Roughly once every few days in order to keep all of our core packages synced with one another.

Code of Conduct

pradyunsg commented 2 years ago

I can confirm that the project has hit the total size limit of 10.0 GiB.

There are 504 releases currently. Each release that I looked at has a roughly 500KB wheel file and a 26 MB source distirbution file.

pradyunsg commented 2 years ago

It's unclear to me what should be done here, since this project's size limit is already at 10 GB (which it has now hit). I'll mark this as awaiting response, once OP is able to provide context on why the source distribution is so big and whether they need an increased limit (if so, please edit the first post and title) or they will delete certain files or change their workflows.

chriswmackey commented 2 years ago

Thank you for the response and for the explanation, @pradyunsg .

I did not realize that the source code of the release was ~25MB since only the .whl file was showing up under the Download Files section of the latest releases. Moreover, I can see that our test suite and our online docs are somehow being included in the .tar.gz source of the releases, which is definitely not our intention. The actual source code itself is only ~1.5 MB when it is fully decompressed and the compressed .tar.gz file is less than half of a MB. I will change our CI to make sure that we only release the source code without the tests going forward.

I would ideally like to go back through our previous releases and replace them with versions that exclude our test suite. If you could point me to any documentation that might help me do a mass update of the sdist in our previous releases, this would be greatly appreciated.

Thank you, again.

chriswmackey commented 2 years ago

Hi @pradyunsg ,

I just wanted to confirm that I have edited our automated deployment to exclude all tests and docs. This has dropped the size of our sdist from ~25 MB down to 0.33 MB. Furthermore, I have gone through the honeybee-energy release history and deleted ~50 of the oldest releases in order to free up some space. I am performing similar exclusions of tests and docs on our other PyPI packages. So you should not have any issues with us pushing large distributions in future releases.

My only concern now is that we have a lot of large recent releases of honeybee-energy, which could mean that we eventually run out of space again in the future. From what I can tell, it does not seem possible to edit the sdist of previous releases. However, it looks like I can delete the sdist while still leaving the .whl file. Given that the latest releases of honeybee-energy seem to work with only the .whl file and not the sdist, it seems like deleting the sdist of the latest releases could be a nice solution.

Can you confirm that deleting the sdist of the latest honeybee-energy releases will still allow our end-users to pip install honeybee-energy (as long as the .whl file remains)?

If so, I'll do that. If not, would it be possible to get a limit extension to 11 GB just for this package? By my calculations, that limit increase should now last us another 5-10 years.

pradyunsg commented 2 years ago

I've bumped it to 15 GB. You shouldn't need to delete any more files. I expect that this limit will work for you, for quite a while. :)

chriswmackey commented 2 years ago

Thank you, @pradyunsg ! That will work very well.