riverqueue / river

Fast and reliable background jobs in Go
https://riverqueue.com
Mozilla Public License 2.0
2.86k stars 68 forks source link

Add basic plugin system for clients #430

Closed brandur closed 1 week ago

brandur commented 1 week ago

Here, follow up #429 to add a basic plugin system for River clients which allows a driver to add maintenance and non-maintenance services to a client before it starts up. The plugin interface is implemented by the drivers themselves, and looks like this:

type driverPlugin[TTx any] interface {
    // PluginInit initializes a plugin with an archetype and client. It's
    // invoked on Client.NewClient.
    PluginInit(archetype *baseservice.Archetype, client *Client[TTx])

    // PluginMaintenanceServices returns additional maintenance services (will
    // only run on an elected leader) for a River client.
    PluginMaintenanceServices() []startstop.Service

    // PluginServices returns additional non-maintenance services (will run on
    // all clients) for a River client.
    PluginServices() []startstop.Service
}

The change is fairly straightforward, and we make sure to bring in some test cases verifying the plugin services were indeed added correctly.

brandur commented 1 week ago

Hold on review on this. Needs more work.

brandur commented 1 week ago

@bgentry As mentioned before, I was having trouble finding a good spot to put Plugin because especially with the new rivershared module, I was getting dependency cycles.

I want to pitch a slightly alternative design here where the Plugin type goes away in favor of the driver itself optionally implementing plugin-like functions:

type driverPlugin[TTx any] interface {
    // PluginInit initializes a plugin with an archetype and client. It's
    // invoked on Client.NewClient.
    PluginInit(archetype *baseservice.Archetype, client *Client[TTx])

    // PluginMaintenanceServices returns additional maintenance services (will
    // only run on an elected leader) for a River client.
    PluginMaintenanceServices() []startstop.Service

    // PluginServices returns additional non-maintenance services (will run on
    // all clients) for a River client.
    PluginServices() []startstop.Service
}

It's a little less elegant maybe, but it has the sizable advantage that no new types are needed anywhere — a plugin can implement it by just referencing rivershared and river root. I prefixed the function names with Prefix so they can group nicely on a driver that's implementing them.

Thoughts?

brandur commented 1 week ago

Thanks!