Feature: Terminate long running builds and tasks

[bomoko]

This introduces functionality to delete long running tasks and builds, the default termination value is 6 hours. This is adjustable via flags. It is enabled by default.

There is the ability to label namespaces with additional labels that can override the default, eventually these will be configurable in the Lagoon API though.

Checklist

• [ ] Affected Issues have been mentioned in the Closing issues section
• [ ] Documentation has been written/updated
• [ ] PR title is ready for changelog and subsystem label(s) applied

Closing issues

closes #182

[shreddedbacon]

Just another thought, but we have seen some people utilising tasks to do things that are somewhat questionable in their duration. Could we possibly allow a namespace to be labelled with something like lagoon.sh/taskPodTimeout=6 and lagoon.sh/buildPodTimeout=6 so that we have a way for specific namespaces to be increased or decreased from the default?

That way we can leave the default low, but for problem namespaces, we can manually add the label to the affected namespaces?

Thoughts?

[bomoko]

sounds good @shreddedbacon - I've added some functionality for that now, as well as starting some unit testing.

[shreddedbacon]

Do you want to change this back to draft, or are you happy where its at now to do a final check over for merge?

[bomoko]

@shreddedbacon - Unless there are any other things we want to get in now, I think we're good for final checks, testing, and merging.