search

ecosyste-ms / packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.
https://packages.ecosyste.ms
GNU Affero General Public License v3.0
26 stars 5 forks source link

Packages data doesn't look like it's been updated in 6 months #943

Open joshbressers opened 1 day ago

joshbressers commented 1 day ago

It looks like the packages data hasn't been updating

If you look at this link https://packages.ecosyste.ms/registries/npmjs.org/packages?order=desc&sort=latest_release_published_at (I made it non clickable so hopefully the bots don't hit it, it's clearly an expensive query)

The top hits say they were last updated 6 months ago. The graphs I have from the packages endpoint seem to back this up

I'm not sure what other details could be useful, if you have question please let me know.

andrew commented 1 day ago

I'm investigating, I think the latest_release_published_at field isn't being set correctly.

andrew commented 1 day ago

Just deploying a fix now, will take a file to update the following fields on all packages: (latest_release_published_at, latest_release_number, first_release_published_at)

andrew commented 1 day ago
Screenshot 2024-11-11 at 23 22 07

Those fields are starting to be updated again now ☝️

I'm off to bed now but will pick it up again in the morning to make sure all records are update, it might take a couple days total to get all 10 million packages processed.

Thanks for reporting

joshbressers commented 1 day ago

Holy cow that was fast, thanks!

andrew commented 1 day ago

So far I've fixed up/updated about 10% of all packages, focusing on the top packages across all ecosystems to begin with.

Also related, I'm running a background script to add extra info to all 50 million npm version records: https://github.com/ecosyste-ms/packages/issues/838 (currently around 25% complete)