grafana / thema

A CUE-based framework for portable, evolvable schema
Apache License 2.0
229 stars 12 forks source link
config cue cuelang grafana logic-programming openapi openapi3 packaging schema versioning

Thema

[!WARNING] This repository is no longer actively maintained or supported. If you are interested in continuing the development of this project, feel free to fork the repository and customize it to your needs.

Thema is a system for writing schemas. Much like JSON Schema or OpenAPI, it is general-purpose and its most obvious application is as an IDL. However, those systems treat changing schemas as out of scope: a single version of a schema for some object is the atomic unit, and versioning is left to opaque strings in external systems like git or HTTP. Thema, by contrast, makes schema change a first-class system property: the atomic unit is the set of schema for some object, iteratively appended to over time as requirements evolve.

Thema's approach is novel, so an analogy to the familiar may help. "Branching by abstraction" suggests that you refactor large applications not with long-running VCS branches and big-bang merges, but by letting old and new code live side-by-side on main, and choosing between them with logical gates, like feature flags. Thema is "schema versioning by abstraction": all versions of a schema live side-by-side on main, within logical structures Thema defines.

This holistic view allows Thema to act like a typechecker, but for change-safety between schema versions: either schema versions must be backwards compatible, or there must exist logic to translate a valid instance of schema from one schema version to the next. CUE, the language in which Thema schemas are written, allows Thema to mechanically verify these properties.

These capabilities make Thema a general framework for decoupling the evolution of communicating systems. This can be outward-facing: Thema's guardrails allow anyone to create APIs with Stripe's renowned backwards compatibility guarantees. Or it can be inward-facing: or to change the messages passed in a mesh of microservices without intricately orchestrating deployment.

Learn more in our docs, or in this overview video! (Some things have been renamed since that video, but the logic is unchanged.)

Usage

Thema defines the way schemas are written, organizing each object's history into a "lineage." Once authored, Thema also provides tools for working with lineages via a few basic operations. There are a few different usage patterns, all largely equivalent in capability:

The CLI and server modes are bundled together in the thema command. To install:

go install github.com/grafana/thema/cmd/thema@latest

Maturity

Thema is a young project. The goals are large, but bounded: we will know when the core system is complete. And it mostly is, now - though some breaking changes to how schemas are written are planned before reaching stability.

It is not yet recommended to replace established, stable systems with Thema, but experimenting with doing so is reasonable (and appreciated!). For newer projects, Thema may be a good choice today; the decision is likely to come down to whether the long-term benefit of a simpler architecture for authoring, composing and evolving schema will offset the short-term cost of some incomplete functionality and breaking changes.

Prior/Related Art

A number of systems partially overlap with Thema - for some data, rolling together a set of schema with the relations between those schema.

[^evaluator]: Using Thema as a library in a language depends on a CUE evaluator for that language. Currently, the only CUE evaluator is written in Go.