search

gnl / playback

Easier-than-print dataflow tracing to tap> and Portal with automatic last-input function replay on eval, instant re-render and effortless extraction of traced data
Other
93 stars 1 forks source link
babashka clojure clojurescript debugging interactive-development portal repl tracing

readme

:linkattrs: :sectanchors: ifdef::env-github,env-cljdoc[] :tip-caption: :bulb: :note-caption: :information_source: :caution-caption: :fire: :warning-caption: :warning: endif::[]

image:https://img.shields.io/clojars/v/com.github.gnl/playback.svg[Clojars Project,link=https://clojars.org/com.github.gnl/playback] image:https://img.shields.io/badge/License-EPL%202.0-94A5F5.svg[License,link=https://choosealicense.com/licenses/epl-2.0/]

{empty} +

++++

playback logo

++++

{empty} +

Interactive Programming and Print Debugging Reimagined

Playback provides immediate, frictionless visibility into a program's dataflow, while being simpler to use than print; makes it easy to call and replay functions with real application input; enables the hassle-free extraction of any data observed in the process; and makes possible an exceptionally tight dev loop with instant feedback, including a hot-reload-and-re-render workflow that is much faster than autobuild-on-save.

It does so out of the box with a minimal number of intuitive, unintrusive, https://fishshell.com/docs/current/design.html#configurability-is-the-root-of-all-evil[zero-config] operations. Two and a half reader tags, to be exact:

[source]

> ; trace output + replay function

>> ; trace output and input/bindings/steps (depending on the form) + replay function

>< ; reference traced data

image:doc/images/playback-screenshot.png?raw=true[link="https://vimeo.com/853054487"]

Quick Start and Walkthrough

. Add image:https://img.shields.io/clojars/v/com.github.gnl/playback.svg[Clojars Project,link=https://clojars.org/com.github.gnl/playback] to your dev (and only dev) dependencies. + WARNING: Playback requires Clojure(-Script) version 1.10+.

. Load the playback.preload namespace on application launch and the https://github.com/djblue/portal[Portal] window should pop right up:

. +#>+ traces the evaluated output of any code and sends the data to Portal; + +#>>+ traces input or intermediate data as well, depending on the form (let bindings, threading macro steps, defn/fn arguments, loop/recur bindings, etc.)

. Traced named functions – +#>(defn+, +#>(>defn+, +#>(defmethod+ – as well as registered anonymous functions like re-frame handlers and subscriptions – +#>(reg-event-fx+, +#>(reg-sub+ – cache their most recent input and are automatically called with it (= replayed) whenever they're reloaded by your environment's send-to-REPL or recompile-on-save functionality. + This way you can have a function receive some real application data, make changes to the code, eval/reload it and instantly see the updated dataflow, output and possible spec failures as it's auto-replayed and retraced with the same input. + CAUTION: Playback does not auto-replay functions whose names end with !, so make sure to name your STM-unsafe functions https://guide.clojure.style/#naming-unsafe-functions[as the Gods intended]. + TIP: Use additional nested +#>+ or +#>>+ tags inside the function to zoom in on the specific code you're interested in, similar to the way you would use print statements.

. +#>< _+ references the currently selected data in the Portal window. You can take any previously traced data, feed it to another function, play around with it in the REPL and +#>+ the results back to Portal to inspect them. + NOTE: The _ in +#>< _+ is just a random placeholder, because Clojure reader tags have to be applied to a form. You can use anything in its place and it will be replaced with the selected Portal data. I like +#><[]+.

TIP: Playback works great with Fulcrologic's indispensable https://github.com/fulcrologic/guardrails[Guardrails] (or its now discontinued predecessor https://github.com/gnl/ghostwheel[Ghostwheel]). If you set {:tap>? true} in https://github.com/fulcrologic/guardrails#configuration[its configuration], you can see the results of failed spec checks in Portal right inside the traced dataflow (though you'll probably still want to take a look at the REPL or console for the human-readable error message). A similar workflow should be possible with https://github.com/metosin/malli/blob/master/docs/function-schemas.md[Malli function schemas] and instrumentation as well.

[TIP]

To select a particular Portal viewer for a traced expression, you can define some helper functions like this:

.dev/user.clj(s) [source,clojure]

(ns user)

(defn tree [expr] (with-meta expr {:portal.viewer/default :portal.viewer/tree}))

(defn table [expr] (with-meta expr {:portal.viewer/default :portal.viewer/table}))

You can then wrap the expression with #>> (user/table (...)). See https://cljdoc.org/d/djblue/portal/0.48.0/doc/ui-concepts/viewers[the Portal documentation] for more details.

[TIP]

If you want to trace/replay your own (compatible) macros with Playback, you can extend its existing optypes like this:

.dev/user.clj [source,clojure]

(ns user (:require [playback.core :as playback]))

(playback/extend-default-optypes! {::playback/defn ['foo.bar/defn 'foo.bar/defn-] ::playback/fn-reg ['foo.bar/reg-sub 'foo.bar/reg-fx]})

See +optype->ops+ in playback.core for a list.

Instant re-render on eval

To get faster feedback and more control over hot-reloading, it's recommended that you disable autobuild and instead use your editor's send-top-form-to-REPL functionality in combination with Playback's refresh-on-eval middleware to manually reload individual functions.

. Add the middleware: + .shadow-cljs.edn [source,clojure]

{:nrepl {:middleware [playback.nrepl-middleware/refresh-on-eval] :port 9000}}

+ See the https://shadow-cljs.github.io/docs/UsersGuide.html#nREPL[Shadow documentation] for more details or the https://nrepl.org/nrepl/usage/server.html[nREPL one] for info on other build tools.

. Initialise it on launch: + .dev/user.clj (<- not .cljs) [source,clojure]

(ns user (:require [playback.nrepl-middleware :as middleware]))

(middleware/init-refresh-on-eval! ;; Refresh/re-render functions to call post-reload ['gnl.clojure-playground.main/mount-root] ;; Namespace prefixes in which eval triggers a refresh ["gnl.clojure-playground"])

. Disable autobuild: + .Shadow REPL [source,clojure]

;; In the Clojure REPL, before starting the ClojureScript one with (shadow/repl :app): (shadow/watch-set-autobuild! :app false) ;; To trigger a manual recompile: (shadow/watch-compile! :app)

WARNING: By default, refresh-on-eval is disabled for traced functions, the idea being that you would usually mess around in the code, repeatedly sending it to the REPL to replay and watch the dataflow in the trace, rinse and repeat until it works, and only then would you remove the +#>+ tag, reload and have the application re-render. You can change this behaviour with (middleware/set-refresh-on-traced-fn! true).

TIP: If you are using a Clojure REPL in a namespace with a refresh-enabled prefix meant for ClojureScript, the middleware will try to call the likely non-existent Clojure equivalent of the re-render function and throw an exception. The simplest solution is to create a noop function with the same name that doesn't do anything.

On using (unqualified) reader tags

Unqualified, non-namespaced reader tags are reserved for Clojure and their usage by anyone else is https://clojure.org/reference/reader#tagged_literals[frowned upon] by the powers that be, and for a good reason. That being said, I went ahead, did it anyway and – in the time-honoured tradition of everyone who ever thought they knew better while not being in charge – chose to ask for forgiveness rather than permission. This is why:

The Road to 1.0

...in no particular order:

Contributions and Support

I'm always open to PRs, but please do reach out first if you want to tackle something bigger so we can make sure we're on the same page.

Other than that, if you or your company have benefitted professionally from my open-source work or would simply like to support further development and can afford it, your GitHub sponsorship would be much appreciated:

https://github.com/sponsors/gnl[*Become a Champion of the Lisp Arts*]

General inquiries as to my availability for paid work, open source or otherwise, are welcome.

Acknowledgements, Prior Art and Rationale

First the obligatory disclaimer that Playback stands on the shoulders of giants – those being https://github.com/philoskim/debux[Philos Kim's debux] and https://github.com/djblue/portal[Chris Badahdah's portal] in particular – and mostly just does some dot-connecting and magic-sprinkling on top in order to fuse them into what is hopefully a highly enjoyable interactive development experience, for which, as my small contribution to the never-ending abuse of the REPL acronym, I would like to propose the term RETL, as in Read–Eval–Trace Loop.

The idea to re-render on eval was stolen from https://github.com/mkarp/cljs-nrepl-exercise[Misha Karpenko's nREPL experiments]; https://github.com/spellhouse/clairvoyant[Spellhouse's Clairvoyant] and https://github.com/day8/re-frame-tracer[Day8's re-frame tracer] were the initial inspiration for and the foundation of https://github.com/gnl/ghostwheel#evaluation-tracing-and-program-observability[Ghostwheel's tracing functionality] which was a first shaky step towards what I imagined REPL-based development and debugging should more or less look like. The https://github.com/gnl/ghostwheel#rationale[corresponding section] of the original omnibus project's README is a good summary of the evolving vision that Playback is a part of.

https://github.com/jpmonettas/flow-storm-debugger[Juan Monetta's FlowStorm] is a fantastic tracing debugger that fits perfectly within this vision, but appears to occupy a somewhat different category than Playback – one in which a certain level of (relative) complexity is considered a reasonable trade-off for maximum capability. Playback meanwhile aims to extract the highest possible amount of power from the constraints of not exceeding the complexity of print. I believe it actually manages to be even simpler than that and is therefore not a trade-off. Depending on the situation, sometimes exchanging simplicity for power is worth it and sometimes it is not – and Playback's success as a debugging tool is measured by whether you instinctively reach for it instead of print in the latter case.

But to look at it as just a type of debugger, tracer or dataflow inspector is to sell it short. In combination with https://github.com/fulcrologic/guardrails[Guardrails] or https://github.com/metosin/malli/blob/master/docs/function-schemas.md[Malli function schemas] in particular, it provides instant, precise feedback on the type, content and rendering of real application data repeatedly flowing through a function as it changes iteratively in a tight, low-latency dev loop largely free of many of the common challenges and pitfalls of REPL workflows or dynamically typed languages in general, for that matter. It reduces the extensive amount of mental code compilation and execution that developers commonly perform in their heads, by a significant enough amount that it can be reasonably considered to be a different, and better, paradigm, one that gets much closer to fulfilling the interactive programming promise that classical REPL-based development often fails to deliver on.

I believe we have some low-hanging Clojure fruit to pick here and this is the way.

As always, go boldly forth, fellow maker, create freely and be not afraid of a messy road.

{empty} + Copyright (c) 2023 George Lipov + Licensed under the https://choosealicense.com/licenses/epl-2.0/[Eclipse Public License 2.0]