search

framed-data / overseer

Overseer is a library for building and running data pipelines in Clojure.
Eclipse Public License 1.0
97 stars 10 forks source link

Pass :pre-process return into process #21

Closed andrewberls closed 9 years ago

andrewberls commented 9 years ago

Work was done in https://github.com/framed-data/overseer/pull/20 to implement a 'harnessing' system, which allowed for finer-grained instrumentation of the handler stage functions. However, this system was implemented stemming from the need to transform input to a job prior to execution, which is what a pre-processor is perfect for, of course.

The handler stage signatures are much more coherent when the default pre-processor passes its input to :process, and :post-process returns its result unmodified.

danevans commented 9 years ago

This looks like a good improvement to me :+1: