API | Wiki | Latest releases | Slack channel
Structured telemetry library for Clojure/Script
Telemere is a pure Clojure/Script library that offers an elegant and simple unified API to cover:
- Traditional logging (string messages)
- Structured logging (rich Clojure data types and structures)
- Events (named thing happened, with optional data)
- Tracing (nested flow tracking, with optional data)
- Basic performance monitoring (nested form runtimes)
- Any combination of the above
It's small, super fast, easy to learn, easy to use, and absurdly flexible.
It helps enable Clojure/Script systems that are easily observable, robust, and debuggable - and it represents the refinement and culmination of ideas brewing over 12+ years in Timbre, Tufte and Truss.
See here for full introduction (concepts, terminology, getting started).
Latest release/s
Next-gen observability
A key hurdle in building observable systems is that it's often inconvenient and costly to get out the kind of detailed info that we need when debugging.
Telemere's strategy to address this is to:
- Provide lean, low-fuss syntax to let you conveniently convey program state.
- Use the unique power of Lisp macros to let you dynamically filter costs as you filter signals (pay only for what you need, when you need it).
- For those signals that do pass filtering: move costs from the callsite to a/sync handlers with explicit threading and back-pressure semantics and performance monitoring.
The effect is more than impressive micro-benchmarks. This approach enables a fundamental (qualitative) change in one's approach to observability.
It enables you to write code that is information-verbose by default.
Quick examples
(Or see examples.cljc for REPL-ready snippets)
Create signals
```clojure
(require '[taoensso.telemere :as t])
;; (Just works / no config necessary for typical use cases)
;; Without structured data
(t/log! :info "Hello world!") ; %> Basic log signal (has message)
(t/event! ::my-id :debug) ; %> Basic event signal (just id)
;; With structured data
(t/log! {:level :info, :data {...}} "Hello again!")
(t/event! ::my-id {:level :debug, :data {...}})
;; Trace (can interop with OpenTelemetry)
;; Tracks form runtime, return value, and (nested) parent tree
(t/trace! {:id ::my-id :data {...}}
(do-some-work))
;; Check resulting signal content for debug/tests
(t/with-signal (t/event! ::my-id)) ; => {:keys [ns level id data msg_ ...]}
;; Getting fancy (all costs are conditional!)
(t/log!
{:level :debug
:sample-rate 0.75 ; 75% sampling (noop 25% of the time)
:when (my-conditional)
:rate-limit {"1 per sec" [1 1000]
"5 per min" [5 60000]}
:rate-limit-by my-user-ip-address ; Optional rate-limit scope
:do (inc-my-metric!)
:let
[diagnostics (my-expensive-diagnostics)
formatted (my-expensive-format diagnostics)]
:data
{:diagnostics diagnostics
:formatted formatted
:local-state *my-dynamic-context*}}
;; Message string or vector to join as string
["Something interesting happened!" formatted])
```
Filter signals
```clojure
;; Set minimum level
(t/set-min-level! :warn) ; For all signals
(t/set-min-level! :log :debug) ; For `log!` signals only
;; Set namespace and id filters
(t/set-ns-filter! {:disallow "taoensso.*" :allow "taoensso.sente.*"})
(t/set-id-filter! {:allow #{::my-particular-id "my-app/*"}})
;; Set minimum level for `event!` signals for particular ns pattern
(t/set-min-level! :event "taoensso.sente.*" :warn)
;; Use middleware to:
;; - Transform signals
;; - Filter signals by arb conditions (incl. data/content)
(t/set-middleware!
(fn [signal]
(if (-> signal :data :skip-me?)
nil ; Filter signal (don't handle)
(assoc signal :passed-through-middleware? true))))
(t/with-signal (t/event! ::my-id {:data {:skip-me? true}})) ; => nil
(t/with-signal (t/event! ::my-id {:data {:skip-me? false}})) ; => {...}
;; See `t/help:filters` docstring for more filtering options
```
Add handlers
```clojure
;; Add your own signal handler
(t/add-handler! :my-handler
(fn
([signal] (println signal))
([] (println "Handler has shut down"))))
;; Use `add-handler!` to set handler-level filtering and back-pressure
(t/add-handler! :my-handler
(fn
([signal] (println signal))
([] (println "Handler has shut down")))
{:async {:mode :dropping, :buffer-size 1024, :n-threads 1}
:priority 100
:sample-rate 0.5
:min-level :info
:ns-filter {:disallow "taoensso.*"}
:rate-limit {"1 per sec" [1 1000]}
;; See `t/help:handler-dispatch-options` for more
})
;; See current handlers
(t/get-handlers) ; => { {:keys [handler-fn handler-stats_ dispatch-opts]}}
;; Add console handler to print signals as human-readable text
(t/add-handler! :my-handler
(t/handler:console
{:output-fn (t/format-signal-fn {})}))
;; Add console handler to print signals as edn
(t/add-handler! :my-handler
(t/handler:console
{:output-fn (t/pr-signal-fn {:pr-fn :edn})}))
;; Add console handler to print signals as JSON
;; Ref. (or any alt JSON lib)
#?(:clj (require '[jsonista.core :as jsonista]))
(t/add-handler! :my-handler
(t/handler:console
{:output-fn
#?(:cljs :json ; Use js/JSON.stringify
:clj jsonista/write-value-as-string)}))
```
Why Telemere?
Ergonomics
- Elegant, lightweight API that's easy to use, easy to configure, and deeply flexible.
- Sensible defaults to make getting started fast and easy.
- Extensive beginner-oriented documentation, docstrings, and error messages.
Interop
Scaling
- Hyper-optimized and blazing fast, see performance.
- An API that scales comfortably from the smallest disposable code, to the most massive and complex real-world production environments.
- Auto handler stats for debugging performance and other issues at scale.
Flexibility
- Config via plain Clojure vals and fns for easy customization, composition, and REPL debugging.
- Unmatched environmental config support: JVM properties, environment variables, or classpath resources. Per platform, or cross-platform.
- Unmatched filtering support: by namespace, id pattern, level, level by namespace pattern, etc. At runtime and compile-time.
- Fully configurable a/sync dispatch support: blocking, dropping, sliding, etc.
- Turn-key sampling, rate-limiting, and back-pressure monitoring with sensible defaults.
Comparisons
Videos
Lightning intro (7 mins):
REPL demo (24 mins):
API overview
See relevant docstrings (links below) for usage info-
Creating signals
Name |
Signal kind |
Main arg |
Optional arg |
Returns |
log! |
:log |
msg |
opts /level |
Signal allowed? |
event! |
:event |
id |
opts /level |
Signal allowed? |
error! |
:error |
error |
opts /id |
Given error |
trace! |
:trace |
form |
opts /id |
Form result |
spy! |
:spy |
form |
opts /level |
Form result |
catch->error! |
:error |
form |
opts /id |
Form value or given fallback |
signal! |
<arb> |
opts |
- |
Depends on opts |
Internal help
Detailed help is available without leaving your IDE:
Performance
Telemere is highly optimized and offers great performance at any scale, handling up to 4.2 million filtered signals/sec on a 2020 Macbook Pro M1.
Signal call benchmarks (per thread):
Compile-time filtering? |
Runtime filtering? |
Profile? |
Trace? |
nsecs / call |
✓ (elide) |
- |
- |
- |
0 |
- |
✓ |
- |
- |
350 |
- |
✓ |
✓ |
- |
450 |
- |
✓ |
✓ |
✓ |
1000 |
- Nanoseconds per signal call ~ milliseconds per 1e6 calls
- Times exclude handler runtime (which depends on handler/s, is usually async)
- Benched on a 2020 Macbook Pro M1, running Clojure v1.12 and OpenJDK v22
Performance philosophy
Telemere is optimized for real-world performance. This means prioritizing flexibility and realistic usage over synthetic micro-benchmarks.
Large applications can produce absolute heaps of data, not all equally valuable. Quickly processing infinite streams of unmanageable junk is an anti-pattern. As scale and complexity increase, it becomes more important to strategically plan what data to collect, when, in what quantities, and how to manage it.
Telemere is designed to help with all that. It offers rich data and unmatched filtering support - including per-signal and per-handler sampling and rate-limiting, and zero cost compile-time filtering.
Use these to ensure that you're not capturing useless/low-value/high-noise information in production! With appropriate planning, Telemere is designed to scale to systems of any size and complexity.
See here for detailed tips on real-world usage.
Included handlers
See ✅ links below for features and usage,
See ❤️ links below to vote on future handlers:
You can also easily write your own handlers.
Community
My plan for Telemere is to offer a stable core of limited scope, then to focus on making it as easy for the community to write additional stuff like handlers, middleware, and utils.
See here for community resources.
Documentation
Funding
You can help support continued work on this project, thank you!! 🙏
License
Copyright © 2023-2024 Peter Taoussanis.
Licensed under EPL 1.0 (same as Clojure).