search

sethmlarson / pypi-data

Data about packages and maintainers on PyPI
Apache License 2.0
122 stars 8 forks source link

2023-03-19 does not have db attached #22

Closed konstin closed 1 year ago

konstin commented 1 year ago

seems like the actual db got lost in the 2023-03-19 release:

image

sethmlarson commented 1 year ago

Oh no! I see what happened, the database is over the 150MB limit of GitHub even when compressed. That's unfortunate :/

konstin commented 1 year ago

maybe a different compression such as zstd could push it down below 150MB again

jefftriplett commented 1 year ago

I was here to note the same thing. This might motivate me to add proper django packages integration instead of pulling your data file once a month 😄

hugovk commented 1 year ago

Where does the 150 MB limit come from?

https://docs.github.com/en/repositories/releasing-projects-on-github/about-releases#storage-and-bandwidth-quotas says:

Each file included in a release must be under 2 GB. There is no limit on the total size of a release, nor bandwidth usage.

Is this a different thing?

sethmlarson commented 1 year ago

Screenshot from 2023-03-28 09-49-29

This is the error I'm seeing when trying to upload the file, the error message is not helpful so I'm not really certain that it's a file size issue. Could be anything, really?

sethmlarson commented 1 year ago

Hmm, I'm doubtful it's the file size now, looking in Firefox inspection the request is failing with a CORS error? Very strange :/

sethmlarson commented 1 year ago

Welp, the exact same action just succeeded in Chrome, I was using Firefox the whole time trying to upload the file. So maybe this is a web compatibility issue... :fearful: Either way, the pypi.db.gz is now available for the release! :rocket:

https://github.com/sethmlarson/pypi-data/releases/tag/2023.03.19