search

nodejs / unofficial-builds

Unofficial binaries for Node.js
https://unofficial-builds.nodejs.org
221 stars 46 forks source link

Add ability to trigger new builds of specific recipes #114

Open rvagg opened 5 months ago

rvagg commented 5 months ago

We have this problem with running re-builds of old versions: https://github.com/nodejs/docker-node/issues/1853

i.e. we have the ability to requeue builds of any version, but that will rebuild all assets and they'll get new binaries and new hash digests in SHASUMS which lead to verification problems when anyone tries to check that the binaries included in a Docker image that depends on these binaries don't match the digests that we publish.

But, there's regularly reasons we want to queue builds of old versions, such as when we add a new recipe and want binaries for an old version (or just want to try it out on the most recent version); or when a recipe is broken for some version but gets fixed.

I'm not sure how this would work because currently we just have a version that passes through the queue, perhaps we have <version>[ recipe[, recipe[...]]] in the queue and pass that along and when recipes are specified we only process those.

matthewkeil commented 5 months ago

It looks like the current shape of the queue file is just the version. Just to check, you would like to add the recipes in comma separated format after a space.

v20.1.3 x64-debug,arml6l,musl
v21.2.3
v21.2.4-rc.1

where anything after the space would be interpreted as a recipe and in cases where there are none all recipes get built?

I can take a look at building that functionality this this week.... Or at least give a go at it

rvagg commented 5 months ago

I'm not tied to a particular implementation of this, just thinking aloud given the current form, if you come up with a better idea then that's fine too but I guess keeping complexity low would be a worthy goal.

When I was in build.sh yesterday I was thinking that maybe it should be split out to just build individual recipes, and then coordinate them elsewhere. Maybe even the queueing mechanism is responsible for iterating over the recipe list and then the queue processor is just a dumb receiver that works through its list. The only complication would be that they all need fetch-source to be run first, so there'd need to be a check "has the source for this version been fetched?" before each run.

We also have this in the TODO list https://github.com/nodejs/unofficial-builds/issues/22 and that could potentially be bundled up with this work. The builds are now taking 10h and the new x64-debug is going to extend that more; so it's a long wait if your build is done but in staging.