petemounce
commented
6 years ago Ah, yes. We have some desires around prioritisation and queuing too. I haven't figured out what the use-case for allowing an agent to be a higher or lower priority, yet (could be an improvement in the docs to add some examples?).
Some of our use-cases:
- We want to be able to have jobs that affect production run as soon as possible (which, at this point, does not mean interrupting another job in-flight, since you license per user and not per build or agent, we can just spin up more agents if we need to expedite like this)
- We want to be able to prioritise based on the branch.
- In a master-branch strategy, we'd like
masterbuilds to run with highest priorityhotfix/*builds next (i.e. someone's patching production because incident)feature/*bugfix/*builds next (i.e. general development activity)
- In a long-term-support branch strategy (1) below, we'd like
in-flight/*release activity branches to run firsthotfix/*next (same as above)releases/*next- and here, since we have (say)
10.4.4LTS (low traffic, high importance) up through12.2.1,13.0.1,14x, we'd like to fine-grain deeper
- and here, since we have (say)
feature/*andbugfix/*next (same as above)
- In a master-branch strategy, we'd like
(1) We have a very strong requirement to be able to produce historical builds for versions of our software that could date back years. The reason for this is that our software is designed to underpin games as well as other things, and the games development cycle is measured in years. The appetite for taking on breaking changes (or indeed any changes) of dependencies within games, especially within crunch periods, is basically zero.
So, if we need to fix an issue in a version that's 3 years old, we need to be able to give strong guarantees that the new version contains only the change that the customer needs, so as to reduce the risk to the customer for taking that change.
This translates to doing that thing that almost everyone who uses source control says you should never under any circumstances do ever because it's really hard, and maintain long-lived branches that we release from.
How we do that is the topic of a forthcoming blog post, but suffice to say that we do, and we've thought about it quite hard, and concluded that we do really need to. (We get asked WTF a lot ;) )
kilianc
As we currently have it we can set priority of a node to take a job but not priority of a pipeline in the queue. It would be amazing to be allows to set pipelines as higher priorities than others from time to time.
The reason for this comes from a recent experience we were having at my work. We have 3 iOS build machines running our CI. Our CI runs pretty much at every merge into our repository to ensure high code quality and deployments as needed. We also us the same machines for release builds and deployment to App Store. We came across a high load day for our build machines and our release was going out the same day.
The issue arose that we ended up having 10 or more builds queued and waiting while we also would have somewhere in there the release build. It would be great to be able to give the release build a higher priority than the other builds on those machines so the release build became next in the queue.
The temporary solution would be to have a node only for releases, but that isn't the most feasible because it requires us to either manually make a machine release only for a day, or leave a machine as releases only builds.
As I would see it, we would really need a simple low, medium, and high priority indicator that would allow the queue to be resorted as jobs get added. I do see that some people could see some jobs could in theory never get run, but that would possibly mean poor management on our side and not Buildkite's.
Thanks for taking the time to read this. Cheers :)