Dynamically set timeouts from job

[WearyMonkey]

npm-modules can take a long time to install, e.g. 5 minutes. We solve this by caching the installed modules. We works for 95% of jobs, but for the 5% when the cache is invalid, the job time increases dramatically.

We want to set or job timeouts as tight as possible for the majority case, so that we become aware of performance problems quickly. However this means any job that busts the npm module cache times out.

It would be great if we could set the timeout from the job script, e.g. something like:

if [[ -z "${NPM_CACHE}" ]]; then
  buildkite-agent set-timeout 10m
fi

Or better yet, the ability to temporarily disable the timeout.

e.g.

if [[ -z "${NPM_CACHE}" ]]; then
  buildkite-agent set-timeout 0
  npm install
  buildkite-agent set-timeout 2m
fi

[petemounce]

We'd be able to use this too. Our use-case would be that we run CI within a container that we build from source control. An agent node, when freshly started up, will build this, so have a dramatically higher duration than one that has previously built it.

If BK does any build-duration prediction, it would be handy to incorporate these "hints" to that prediction algorithm.

[joscha]

any chance for this to happen @toolmantim?

[toolmantim]

👋🏼 Have you thought about flipping the problem on its head, and extending the default timeout to handle cold caches but then adding your own timeout command to the thing you want to have a more strict timeout on?

#!/bin/bash

set -e

npm install

timeout 2m my-command