Packages data doesn't look like it's been updated in 6 months

[joshbressers]

It looks like the packages data hasn't been updating

If you look at this link https://packages.ecosyste.ms/registries/npmjs.org/packages?order=desc&sort=latest_release_published_at (I made it non clickable so hopefully the bots don't hit it, it's clearly an expensive query)

The top hits say they were last updated 6 months ago. The graphs I have from the packages endpoint seem to back this up

I'm not sure what other details could be useful, if you have question please let me know.

[andrew]

I'm investigating, I think the latest_release_published_at field isn't being set correctly.

[andrew]

Just deploying a fix now, will take a file to update the following fields on all packages: (latest_release_published_at, latest_release_number, first_release_published_at)

[andrew]

Those fields are starting to be updated again now ☝️

I'm off to bed now but will pick it up again in the morning to make sure all records are update, it might take a couple days total to get all 10 million packages processed.

Thanks for reporting

[joshbressers]

Holy cow that was fast, thanks!

[andrew]

So far I've fixed up/updated about 10% of all packages, focusing on the top packages across all ecosystems to begin with.

Also related, I'm running a background script to add extra info to all 50 million npm version records: https://github.com/ecosyste-ms/packages/issues/838 (currently around 25% complete)

[andrew]

Progress update: 15% of all packages updated, i had to slow the rate of background jobs down a little as it was overwhelming the queue but it's still ticking though.

[andrew]

Progress update: 25% of all packages updated

[andrew]

Progress update: 37% of all packages updated

[andrew]

Finished processing #838 so now I can ramp up on this one, currently at 40%