Tracing flows for debugging

[countvajhula]

The probe debugger allows us to see the values at any point in the flow. This is indispensable for debugging specific issues. But sometimes, we may just want to understand the flow, and we'd like to trace the entire sequence of values when given test inputs. This is analogous to using a dye in biology to see the movements of proteins and microorganisms, or using a test charge to map out an electric field.

In a flow like this one:

(~> sqr (-< add1 sub1) *)

... it would be nice if we could do:

(define-flow do-stuff
  (trace (~> sqr (-< add1 sub1) *)))

... so that it would produce something resembling:

> (do-stuff 3)
(3)
(9)
(10 8)
(80)

Of course, there are actually 6 flows in all here (the entire one, each component of the thread, and each component of the tee), so there is the question of what to do in the case of nested flows. Initially, it should be fine to couple trace to ~> specifically, and have it operate just at the level of the toplevel components of that threading form. Ideally, it might be better to traverse to nested uses of ~> as well, and represent the trace as a tree or DAG.

Just like probe, trace should work even if the flow has an error, showing the trace up to the point where the error is hit. It could even do so without raising the actual error. After all, the trace has not run into an error -- only the flow that is being traced. It could present the trace graphically (even if in text form) to the effect of "here is your problem, at this point in the flow, X, and this is what the error message said."

Tangentially, this is similar to "state monad".

See also: Pipeline profiler

[benknoble]

It would be extra nice if, like #lang debug racket, the output was something like

(~> …) = (3)
sqr = (9)
(-< add1 sub1) = (10 8)
* = (80)