RFC: Simplify API and allow for an adapter pattern

[ScopeyNZ]

This is a pretty wide scope issue, but I believe it should be captured. Currently this module is a bit of a go-to for scheduled tasks. With more adoption in larger scale environments this module is struggling to be fit for purpose - specifically around multiple server environments where creating mutexes becomes difficult. There are many bits of software/organisations that aim to solve this problem (AMQP). Unfortunately the API of queued jobs is quite public and not well defined.

We should:

• Refactor the API of queued jobs (ie. queueJob()) to be a little more expressive with a well defined/documented public interface
• Update the interface for a Job with a similar goal
• Provide some sort of upgrade process for existing jobs. It's not unrealistic that jobs will need to be re-written considering API changes. Refactoring jobs should be relatively painless.
• Work in an adapter pattern for queues allowing them to be handled by database, redis, AMQP etc...
• Stretch goal / discussion needed - Potentially add an adapter for queue runners? Some might be quite simple and not need authentication bootstrapped (like static publishing?)

Keen for some feedback.

[sminnee]

Related: https://github.com/silverstripe/silverstripe-framework/issues/6524

[ScopeyNZ]

Thanks for the heads up @sminnee I didn't even know that RFC existed 😖