0.4.0: First implementation of harnessing

[andrewberls]

Straight from the docs:

A mechanism to 'wrap' job handlers, giving one the ability to provide additional context prior to execution.

Accepts a standard job handler (map or function) and a wrapper function which will be called with the function specified in your handler and is expected to return a new function with the same signature. If your handler is a map, it will be transparently constructed/deconstructed; harnesses work solely in terms of functions.

For example, a harness that simply implements the default behavior is:

  (defn my-harness [f]
    (fn [job]
      (f job)))

A more substantive harnesses can be used to provide jobs with additional context, for example a database connection:

  (defn my-harness [f]
    (fn [job]
      (-> job
          (assoc :conn (d/connect my-datomic-uri))
          (f))))

In the job-handlers map, one specifies

  {:my-job (overseer.api/harness my-job/run my-harness)}

then within your handler:

  (defn run [{:keys [conn] :as job}] ...)

If your handler is a map, you can optionally specify a key to harness a specific stage; the default is :process. To harness a post-processor:

  {:my-job (overseer.api/harness my-job/run :post-process my-harness)}

[elliot42]

:+1:

(Though the test coverage is a bit less than the scope of the functionality, I think)

Although for comparison, I think it's worth reading and considering the following:

https://github.com/pedestal/docs/blob/master/documentation/service-interceptors.md

Basically what we've implemented here is almost exactly equivalent to Ring middlewares, so you can take everything in the above document and apply it to what we're doing here

[elliot42]

:+1: although the test I don't think proves exactly what it says it proves

[andrewberls]

@elliot42 I think this becomes redundant and can be scrapped in favor of better defaults for processors as in https://github.com/framed-data/overseer/pull/21 - we can write a default pre-processor that will emulate the current system/harness. What do you think?

[andrewberls]

Closing this in favor of #21

[unforced]

Reopening PR so it can be used in https://github.com/framed-data/intake-mixpanel/issues/860

[andrewberls]

:+1: once conflicts

[elliot42]

So, if you read the Pedestal article, I think the point it's trying to make is that function wrapper and :pre and :post hooking are both two separate pipeline composition methods. I think they would argue that you should have one uniform composition method, not two composition methods :pre-hooking AND function composition. I think we're running down a weird path here that can/should be simplified, and we should pause and decide intentionally what the single uniform interface/composition method should be overseer rather than throwing tons of "this OR this OR this" options into the system.