search

nodejs / docker-node

Official Docker Image for Node.js :whale: :turtle: :rocket:
https://hub.docker.com/_/node/
MIT License
8.13k stars 1.95k forks source link

Minor change in unofficial-builds workflow #2031

Closed rvagg closed 4 months ago

rvagg commented 4 months ago

Just a quick check with this team to make sure that we're not going to break anything for you. @matthewkeil is working on a new feature for unofficial-builds over at https://github.com/nodejs/unofficial-builds/pull/116 that lets us rebuild targeted recipes + versions (currently just versions and rebuilding old versions causes havoc over here with changes SHASUMS digests). As part of that there's a change that would promote builds as soon as they are ready. Currently all builds have to finish before they get promoted and listed in the SHASUMs.

I'm seeing in https://github.com/nodejs/docker-node/blob/91f6e50dd988db0f39bbcb519f0e8a15ab13f0d4/build-automation.mjs#L62C1-L70C1 where it seems to be used that you're looking for the version in https://unofficial-builds.nodejs.org/download/release/index.json but then doing a directory listing for musl. Can I suggest a simplification - if you are grabbing index.json then you already have the information you need, if it has "linux-x64-musl" in the "files" array for a version then it's in there (this is generated by looking at the files so it's safe unless there's some weird cache issue but you have the same problem with a directory listing). There's also the SHASUMS file which would also be reliable (e.g. https://unofficial-builds.nodejs.org/download/release/v21.0.0/SHASUMS256.txt), but the main point is that a single fetch should be all that's needed.

With this change, index.json should be updated with each promotion so a new version may show up, but linux-x64-musl may not be in it. I have suggested that we may want to prioritise and get certain builds, like linux-x64-musl, out first for the automated consumers, but I don't know whether that even really matters here. Do you have people scrambling to use the builds as soon as possible after upstream release and complaining about the delay (currently >=10h for unofficial-builds)?

SimenB commented 4 months ago

As part of that there's a change that would promote builds as soon as they are ready. Currently all builds have to finish before they get promoted and listed in the SHASUMs.

🎉

a single fetch should be all that's needed.

Very good point! Should be a simple enough change

Do you have people scrambling to use the builds as soon as possible after upstream release and complaining about the delay (currently >=10h for unofficial-builds)?

Yes, quite often (especially for security releases)

SimenB commented 4 months ago

/cc @nodejs/docker FYI

SimenB commented 4 months ago

Opened #2032 which removes the extraneous network call

SimenB commented 4 months ago

@rvagg another thing with the wait from upstream release - when there are multiple releases at once (like #2027 with all 3 supported versions getting a patch) would the priority thing you mention be possible to tweak to build musl for all releases, and then move onto the other builds? Or would it "just" prioritise within a single release line, and then when all are built, move on the the next line?

matthewkeil commented 4 months ago

At the moment @SimenB it is only prioritized within single version. Versions are queued and within a version the individual recipes (musl build of a specific version) are ordered.

rvagg commented 4 months ago

Yeah, that. Any more advanced queueing is going to be stretching the maintainability of the bash-driven workflow I think.