search

cdnjs / api-server

📡 API server for api.cdnjs.com - The #1 free and open source CDN built to make life easier for developers.
https://api.cdnjs.com
MIT License
89 stars 38 forks source link

Latest version problem with fields query param #76

Closed jgazeau closed 2 years ago

jgazeau commented 2 years ago

Help request

Problem

Hello guys, actually I ran into a strange behavior on the cdnjs api. While trying to get the latest version of a library it seems that the response is not always the same when using the fields query param. For example: if you reach this endpoint several times:

https://api.cdnjs.com/libraries/typescript?fields=version,latest

the answer is randomly one of these two responses: {"version":"4.7.4","latest":"https://cdnjs.cloudflare.com/ajax/libs/typescript/4.7.4/typescript.min.js"} {"version":"4.8.0-beta","latest":"https://cdnjs.cloudflare.com/ajax/libs/typescript/4.8.0-beta/typescript.min.js"} Only the second one should be expected

Nevertheless when trying to directly get the full response of a library without the fields query param: https://api.cdnjs.com/libraries/typescript the response is always constant and as expected:

{"name":"typescript","latest":"https://cdnjs.cloudflare.com/ajax/libs/typescript/4.8.0-beta/typescript.min.js","sri":"sha512-
...

Do you know where it can come from ? Kr,

Jordan.

MattIPv4 commented 2 years ago

👀 So it looks like the 4.7.4 version is coming from the new API worker that is currently deployed to 10% of production traffic. The 4.8.0-beta version is coming from the old Express API origin.

Not 100% sure why they're returning different values at present, we are digging.

MattIPv4 commented 2 years ago

In the new API worker we consume the main package metadata to determine the version to return, which does appear to sort the versions using some rudimentary semver logic: https://github.com/cdnjs/tools/blob/2b7cddcb99cae64fca46a0703e548750c7d096c7/functions/kv-pump/main.go#L193 / https://github.com/cdnjs/tools/blob/2b7cddcb99cae64fca46a0703e548750c7d096c7/packages/packages.go#L236-L283

Due to the semver sorting here, it is using 4.7.4 as the latest, even though 4.8.0-beta was processed more recently.

Whereas, the old Express API relied on a slightly different set of aggregated metadata and the logic that generates this doesn't seem to apply any semver sorting, just always using the most recently processed version: https://github.com/cdnjs/tools/blob/2b7cddcb99cae64fca46a0703e548750c7d096c7/kv/aggregate.go#L115

The most recent version to be processed was 4.8.0-beta, so this is the value that is used here.

jgazeau commented 2 years ago

Thanks for the info 😃 . Do you know if it is plan to align the workers API version ? (just to get a consistent behavior)

MattIPv4 commented 2 years ago

The API worker version that's currently rolling out is actually the correct version (aligning with what's served from the main /libraries endpoint for example). Once the rollout is complete, this value will be consistently returning then.

MattIPv4 commented 2 years ago

https://github.com/cdnjs/cdnjs/issues/14140 is tracking that rollout :) (and the PR is #75)

jgazeau commented 2 years ago

Thanks a lot @MattIPv4