Add PyPI integration

[rossmacarthur]

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
spacebadgers	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 10, 2023 10:52pm

[SplittyDev]

Great work, thanks a lot for the contribution!

[SplittyDev]

Apparently the PyPI response is huge (2MB+):

I just had a look at the docs and it doesn't seem like there's much we can do about that, so I guess we'll have to live with PyPI badges taking slightly longer to generate, since they can't be cached on the edge.

There is an RSS endpoint for getting just the releases, which we could theoretically use to fetch only the latest release: https://pypi.org/rss/project/:packageName/releases.xml (numpy example)

But this also includes rc releases and the like, and I'm not sure how to properly differentiate stable releases from release candidates, alpha versions and other variations that probably shouldn't be fetched as latest..

Please let me know if you want me to merge this as-is (which is totally fine with me, we can always come back to it later), or if you wanna hold off on merging for a bit and look into getting the releases and fetching the latest version only.

[SplittyDev]

Off-Topic: I just noticed that you're the creator of sheldon! I've been using that tool for quite some time now, thanks a lot for your valuable contributions to the open source community! ✨

[rossmacarthur]

Apparently the PyPI response is huge (2MB+):

Yeah I did notice this locally as well :/. Comparing with a bunch of other packages I'm seeing a big variation in size e.g. requests is much much smaller.

❯ curl -fsSL -i https://pypi.org/pypi/numpy/json | grep content-length
content-length: 1656988
❯ curl -fsSL -i https://pypi.org/pypi/requests/json | grep content-length
content-length: 163731
❯ curl -fsSL -i https://pypi.org/pypi/scipy/json | grep content-length
content-length: 999371
❯ curl -fsSL -i https://pypi.org/pypi/matplotlib/json | grep content-length
content-length: 1193906
❯ curl -fsSL -i https://pypi.org/pypi/pydantic/json | grep content-length
content-length: 1050036
❯ curl -fsSL -i https://pypi.org/pypi/fastapi/json | grep content-length
content-length: 261320

But this also includes rc releases and the like, and I'm not sure how to properly differentiate stable releases from release candidates, alpha versions and other variations that probably shouldn't be fetched as latest..

I guess we could do this by parsing the version but I'm not sure if Python versions conform to the exact semver format that a Javascript parser can do.

Please let me know if you want me to merge this as-is (which is totally fine with me, we can always come back to it later), or if you wanna hold off on merging for a bit and look into getting the releases and fetching the latest version only.

I'm happy to merge for now since I think most people will not encounter this unless they have many, many releases for their project. Happy to come back to it later.

[rossmacarthur]

Off-Topic: I just noticed that you're the creator of sheldon! I've been using that tool for quite some time now, thanks a lot for your valuable contributions to the open source community! ✨

Glad you found it useful! Thank you for creating spacebadgers, I've been frustrated with https://shields.io for a while now

[SplittyDev]

Maybe we can just replace numpy with a smaller package on the homepage? The homepage is server-side rendered, and if the 2MB responses can't be cached that might significantly add to the loading time for every visitor.

A few examples that we could use instead of numpy:

• docopt-ng
• pickleDB
• janus

I've gone ahead and added the suggested change as a review comment. If you wanna use something else, let me know! I'm not super knowledgeable about the Python community, maybe there's a more iconic package with a smaller size.

As long as it's < 2MB, it should work fine with Vercel's edge cache.

[vercel[bot]]

@rossmacarthur is attempting to deploy a commit to the splittydev's Team Team on Vercel.

A member of the Team first needs to authorize it.

[rossmacarthur]

I've gone ahead and added the suggested change as a review comment. If you wanna use something else, let me know! I'm not super knowledgeable about the Python community, maybe there's a more iconic package with a smaller size.

Interestingly I've noticed that some of the packages I wanted to use don't set the "license" field. E.g. fastapi, seaborn, but set it in the classifiers. Looks like shields.io is extracting it from the classifiers 🤮 https://github.com/badges/shields/blob/master/services/pypi/pypi-helpers.js

[SplittyDev]

Interestingly I've noticed that some of the packages I wanted to use don't set the "license" field. E.g. fastapi, seaborn, but set it in the classifiers. Works on shields.io so I assume they're extracting it from the classifiers.

Ah well, those are a bit annoying to parse. Let's go ahead with merging once CI is green, and I'll do a follow-up for parsing the classifiers to make sure licenses are always working.

[SplittyDev]

@all-contributors please add @rossmacarthur for code

[allcontributors[bot]]

@SplittyDev

I've put up a pull request to add @rossmacarthur! :tada: