Generation of diffs and update functions for ocaml types.
ppx_diff
is a ppx rewriter that generates the implementation of [Diffable.S].
The [Diff.t] type represents differences between two values. The [Diff.get] and
[Diff.apply_exn] functions compute and apply the differences.
Users of ppx_diff
should add the [diffable] library to the dependencies in
[jbuild].
If you define a type as follows:
type t [@@deriving diff]
then code will be generated matching the following signature:
module Diff : sig
type derived_on = t
type t
val get : from : derived_on -> to_ : derived_on -> local_ t Optional_diff.t
val apply_exn : derived_on -> t -> derived_on
val of_list_exn : t list -> local_ t Optional_diff.t
end
Use of [@@deriving diff]
in an .mli will extend the signature with the above module.
In an .ml, definitions will be generated.
You will likely want your diff type to derive some subset of bin_io
, sexp
, of_sexp
,
sexp_of
.
By default [@@deriving diff]
will include the same subset of the above as those
included on your original type.
So e.g. if you define a type as follows:
type t [@@deriving sexp, bin_io, diff]
then code will be generated matching the following signature:
module Diff : sig
type derived_on = t
type t [@@deriving sexp, bin_io]
val get : from : derived_on -> to_ : derived_on -> local_ t Optional_diff.t
val apply_exn : derived_on -> t -> derived_on
val of_list_exn : t list -> local_ t Optional_diff.t
end
If you want to derive additional items on your diff type, you can achieve this using the
extra_derive
argument.
For instance, you can write:
type t [@@deriving sexp_of, diff ~extra_derive:[of_sexp ; bin_io]]
Optional_diff
has the following accessor functions:
module Optional_diff : sig
type 'a t
val is_none : local_ _ t -> bool
val unsafe_value : local_ 'a t -> 'a
val to_option : local_ 'a t -> 'a option
end
It is designed to improve allocations. Specifically, if you have a local_ 'a t
, that
means that 'a
was allocated, but the pointer to 'a option
was not.
Why are the function called apply_exn
and of_list_exn
instead of apply
and of_list
?
Rest assured that calling
Option.iter (Diff.get ~from ~to_ |> Optional_diff.to_option) ~f:(Diff.apply_exn from)
will never raise.
However, applying a hand-crafted diff might raise for some types (in particular for
variants when the diff is for an unexpected variant). Similarly, hand-crafing diffs using of_list_exn
may raise if trying to combine diffs for different variants.
By default, diffs will be "nested". This means if a type deriving diff
references other
types, those types must also be diffable.
For example, for the following to work
type t =
{ x : X.t
; y : Y.t
; z : Z.t
} [@@deriving diff]
X.t
, Y.t
and Z.t
must also either derive diff
or otherwise implement Diffable.S
.
Some types are already supported, so it's ok to reference those. This includes:
int
, float
, string
, bool
, char
, unit
and option
Set
and Map
created using the various Comparable.Make
functorsppx_diff
works also with parametric polymorphic types. In this case the
[Diff.get] and [Diff.apply_exn] functions require diff functions for all the types in order of their
appearance in the type definition. This is similar to what you may be used to with
e.g. t_of_sexp
functions.
The [Diff.of_list_exn]
function requires [of_list_exn]
and [apply_exn]
of all the
types in order of their appearance. (The [apply_exn]
is needed for variant types. If you
combine something like [Set_to_variant_X of a]
with [Diff_variant_X of a_diff]
, the
result should be [Set_to_variant_X (apply_a_exn a a_diff)]
)
If you define a type as follows:
type ('a, 'b) t [@@deriving diff]
then code will be generated matching following signature:
module Diff : sig
type ('a, 'b) derived_on = ('a, 'b) t
type ('a, 'b, 'a_diff, 'b_diff) t
val get
: (from:'a -> to_:'a -> local_ 'a_diff Optional_diff.t)
-> (from:'b -> to_:'b -> local_ 'b_diff Optional_diff.t)
-> from:('a, 'b) derived_on
-> to_:('a, 'b) derived_on
-> local_ ('a, 'b, 'a_diff, 'b_diff) t Optional_diff.t
val apply_exn
: ('a -> 'a_diff -> 'a)
-> ('b -> 'b_diff -> 'b)
-> ('a, 'b) derived_on
-> ('a, 'b, 'a_diff, 'b_diff) t
-> ('a, 'b) derived_on
val of_list_exn
: ('a_diff list -> local_ 'a_diff Optional_diff.t)
-> ('a -> 'a_diff -> 'a)
-> ('b_diff list -> local_ 'b_diff Optional_diff.t)
-> ('b -> 'b_diff -> 'b)
-> ('a, 'b, 'a_diff, 'b_diff) t list
-> local_ ('a, 'b, 'a_diff, 'b_diff) t Optional_diff.t
end
t
ppx_diff
works also with types named something else than t
.
If you define a type as follows
type name [@@deriving diff]
then code will be generated matching the following signature:
module Diff_of_name : sig
type derived_on = name
type t
val get : from : derived_on -> to_ : derived_on -> local_ t Optional_diff.t
val apply_exn : derived_on -> t -> derived_on
val of_list_exn : t list -> local_ t Optional_diff.t
end
Below you can see some examples of what the diff types looks like for various supported types.
The diff of a record is a list of diffs of its fields (and includes only the fields that changed)
type t =
{ x : X.t
; y : Y.t
; z : Z.t
} [@@deriving diff]
module Diff : sig
module Field_diff : sig
type t = | X of X.Diff.t | Y of Y.Diff.t | Z of Z.Diff.t
end
type t = private Field_diff.t list
val create : ?x:X.Diff.t -> ?y:Y.Diff.t -> ?z:Z.Diff.t -> unit -> t
val create_of_variants
: x:local_ (X.Diff.t, Field_diff.t) Of_variant.t
-> y:local_ (Y.Diff.t, Field_diff.t) Of_variant.t
-> z:local_ (Z.Diff.t, Field_diff.t) Of_variant.t
-> t
val singleton : Field_diff.t -> t
...
end
where
module Of_variant : sig
type ('a, 'diff) t = ('a -> 'diff) Variantslib.Variant.t -> local_ 'a Optional_diff.t
end
The Diff.t
type is private, because the list satisfies two invariants:
(@) There is at most one item of each of X
, Y
and Z
variant of Field_diff.t
in t
(@) The items are in sorted order
Those two invariants allow for a more performant apply_exn
function.
You can still create items of the Diff.t
type by calling Diff.singleton
, Diff.create
,
Diff.create_of_variants
or Diff.of_list_exn
Note that Diff.of_list_exn
will correctly order the diffs and combine diffs for the same
field in order to satisfy the invariants.
There is a special case for a record with just one field. We don't return a list, just the (optional) single field diff:
type t = { x : X.t } [@@deriving diff]
module Diff : sig
type t = | X of X.Diff.t
...
end
Very similar to records
type t = X.t * Y.t [@@deriving diff]
module Diff : sig
type t = (X.t, Y.t, X.Diff.t, Y.Diff.t) Diffable.Tuples.Tuple2.Diff.t
...
end
where
module Tuple2 : sig
type ('a1, 'a2) t = 'a1 * 'a2 [@@deriving sexp, bin_io]
module Diff : sig
type ('a1, 'a2) derived_on = ('a1, 'a2) t
module Entry_diff : sig
type ('a1, 'a2, 'a1_diff, 'a2_diff) t =
| T1 of 'a1_diff
| T2 of 'a2_diff
[@@deriving variants, sexp, bin_io]
end
type ('a1, 'a2, 'a1_diff, 'a2_diff) t =
private
('a1, 'a2, 'a1_diff, 'a2_diff) Entry_diff.t list
[@@deriving sexp, bin_io]
val create : ?t1:'a1_diff -> ?t2:'a2_diff -> unit -> ('a1, 'a2, 'a1_diff, 'a2_diff) t
val create_of_variants
: t1:local_ ('a1_diff, ('a1, 'a2, 'a1_diff, 'a2_diff) Entry_diff.t) Of_variant.t
-> t2:local_ ('a2_diff, ('a1, 'a2, 'a1_diff, 'a2_diff) Entry_diff.t) Of_variant.t
-> ('a1, 'a2, 'a1_diff, 'a2_diff) t
val singleton
: ('a1, 'a2, 'a1_diff, 'a2_diff) Entry_diff.t
-> ('a1, 'a2, 'a1_diff, 'a2_diff) t
...
end
end
Again the Entry_diff.t
entries in Diff.t
must be sorted and unique, and again create
,
create_of_variants
, singleton
functions are provided for convenience.
Tuples of size greater than 6
are not supported, but this can be trivially extended as
the Tuple
modules are generated using cinaps.
type t = X.t [@@deriving diff]
module Diff : sig
type t = X.Diff.t
...
end
including parametrized ones
type t = Y.t X1.t [@@deriving diff]
module Diff : sig
type t = (Y.t, Y.Diff.t) X.Diff.t
...
end
type 'a t = 'a [@@deriving diff]
module Diff : sig
type ('a, 'a_diff) t = 'a_diff
...
end
type t =
| A
| B of B.t
[@@deriving diff]
module Diff : sig
type t = | Set_to_a | Set_to_b of B.t | Diff_b of B.Diff.t
...
end
Inlined tuples and records are also supported. Their diff types look exactly like those for regular tuples / records.
type t =
| C of C1.t * C2.t
| D of { d1 : D1.t ; d2 : D2.t}
[@@deriving diff]
module Diff : sig
module D_record : sig
type t =
{ global_ d1 : D1.t
; global_ d2 : D2.t
}
module Diff : sig
type derived_on = t
module Field_diff : sig
type t =
| D1 of D1.Diff.t
| D2 of D2.Diff.t
end
type t = private Field_diff.t list
val get
: from:local_ derived_on
-> to_:local_ derived_on
-> local_ t Optional_diff.t
val apply_exn : local_ derived_on -> t -> local_ derived_on
val of_list_exn : t list -> local_ t Optional_diff.t
val create : ?d1:D1.Diff.t -> ?d2:D2.Diff.t -> unit -> t
val create_of_variants
: d1:local_ (D1.Diff.t, Field_diff.t) Of_variant.t
-> d2:local_ (D2.Diff.t, Field_diff.t) Of_variant.t
-> t
val singleton : Field_diff.t -> t
end
type t =
| Set_to_c of C1.t * C2.t
| Set_to_d of
{ d1 : D1.t
; d2 : D2.t
}
| Diff_c of Diffable.Tuples.Tuple2.For_inlined_tuple.Diff.t
| Diff_d of D_record.Diff.t
...
end
where
type Diffable.Tuples.Tuple2.For_inlined_tuple.Diff.t = Diffable.Tuples.Tuple2.Diff.t
Notice that for inlined records the ppx generates a helper module which looks the same as a regular record
module with [@@deriving diff]
, but can be calculated from / applied to local_
values
of derived_on
.
Similar helper modules exist for tuples in Diffable.Tuples.TupleN.For_inlined_tuple
Since diffs are by default nested, any type you reference must also derive diff
or
otherwise implement Diffable.S
For example, if Some_type
doesn't implement Diffable.S
, then the following:
type t = Some_type.t * Some_type.t
[@@deriving diff]
Will give you an error like:
Error: Unbound module Some_type.Diff
Often it's enough to just add [@@deriving diff]
to the referenced type.
But occasionally that doesn't quite work. This section covers some special cases and how to handle them.
Some types, e.g. any Identifiable
, or indeed Some_type.t
, are super simple, and
what we really want is to have type Diff.t = t
and only return a diff if the two values
aren't equal. We call such diffs atomic
.
This section explains how you can mark your types as atomically diffable
You can annotate any part of your type to use atomic diffs using the [@diff.atomic]
attribute.
For instance, the error from the example above will go away if you write
type t =
(Some_type.t [@diff.atomic]) * (Some_type.t [@diff.atomic])
[@@deriving diff]
The annotation also works for record fields:
type t =
{ start : Some_type.t [@diff.atomic]
; stop : Some_type.t [@diff.atomic]
}
[@@deriving diff]
and for variants:
type t =
| Start of Some_type.t [@diff.atomic]
| Stop of Some_type.t [@diff.atomic]
[@@deriving diff]
The two examples above are in fact equivalent to
type t =
{ start : (Some_type.t [@diff.atomic])
; stop : (Some_type.t [@diff.atomic])
}
[@@deriving diff]
and
type t =
| Start of (Some_type.t [@diff.atomic])
| Stop of (Some_type.t [@diff.atomic])
[@@deriving diff]
but slightly nicer to write.
Finally, you can mark your whole type atomic by using the how
parameter:
type t = Some_type.t [@@deriving equal, diff ~how:"atomic"]
Any type that you mark atomic
, must also derive / otherwise implement equal
.
To avoid having to add the annotations everywhere, you can also make the referenced type
implement Diffable.S
in an atomic fashion.
To do this, use Diffable.Make_atomic
, e.g.
module Id : sig
type t [@@deriving sexp, bin_io]
include Diffable.S with type t := t and type Diff.t = t
end = struct
type t = string [@@deriving sexp, bin_io, equal]
include functor Diffable.Atomic.Make
end
Note that Diffable.Atomic.Make
requires that your type derives equal
, sexp
and bin_io
.
If you don't derive all of those (but do derive, say, sexp_of
), Diffable.Atomic
will
still work but you will need to be more verbose, e.g.
module Id : sig
type t [@@deriving sexp_of]
module Diff : sig
type derived_on = t [@@deriving sexp_of]
type t = derived_on [@@deriving sexp_of]
include Diffable.Diff.S_plain with type t := t and type derived_on := derived_on
end
end = struct
type t = some_type [@@deriving equal, sexp_of]
module Diff = struct
type derived_on = t [@@deriving equal, sexp_of]
type t = derived_on [@@deriving equal, sexp_of]
include functor Diffable.Atomic.Make_diff_plain
end
end
If a type doesn't implement equal
, but does implement compare
, you can also use the
atomic_using_compare
attribute, which works in exactly the same way as atomic
.
E.g. the following will work:
type t = Some_type.t [@@deriving compare, diff ~how:"atomic_using_compare"]
type t =
{ start : Some_type.t [@diff.atomic_using_compare]
; stop : Some_type.t [@diff.atomic_using_compare]
}
[@@deriving diff]
atomic_using_compare
does not work in mlis/signatures, just continue using atomic
there - the generated signature is exactly the same
It may come as a surprise, but in Core the following holds
Float.equal Float.nan Float.nan = false
Float.compare Float.nan Float.nan = 0
If you want to avoid spurious NANs in your diffs, e.g. because you want to manually
inspect your diffs and don't want them too be noisy, or because you expect there to be a
lot of NANs in your values, you need to make sure you use compare
and not equal
In practice this means you'll want to:
atomic_using_compare
and not atomic
Diffable.Atomic.Make
add an override let equal = [%compare.equal]
The primitive float
type is already handled using compare
Sets and maps implemented using the various Make
modules in Comparable
and
Comparable.Stable
already implement Diffable.S
and Diffable.S1
respectively, so you
can use them out of the box.
All of the following will work:
type t = My_id.Set.t [@@deriving diff]
type t = My_id.Stable.V1.Set.t [@@deriving diff]
type t = { value : float My_id.Map.t } [@@deriving diff]
type t = { value : float My_id.Stable.V1.Map.t } [@@deriving diff]
type 'a t = | Value of 'a My_id.Map.t [@@deriving diff]
type 'a t = | Value of 'a My_id.Stable.V1.Map.t [@@deriving diff]
The diff for sets looks like:
module Diff : sig
type derived_on = t
type t = My_id.t Diffable.Set_diff.t
...
end
where
module Diffable.Set_diff : sig
module Change : sig
type 'a t = | Add of 'a | Remove of 'a
end
type 'a t = 'a Change.t list
end
and the diff for maps looks like:
module Diff : sig
type 'v derived_on = 'v t
type ('v, 'v_diff) t = (My_id.t, 'v, 'v_diff) Diffable.Map_diff.t
...
end
where
module Diffable.Map_diff : sig
module Change : sig
type ('key, 'v, 'v_diff) t =
| Remove of 'key
| Add of 'key * 'v
| Diff of 'key * 'v_diff
[@@deriving sexp, bin_io]
end
type ('key, 'v, 'v_diff) t = ('key, 'v, 'v_diff) Change.t list [@@deriving sexp, bin_io]
end
By default the values in a map are also diffed.
So if the following:
type t = Some_type.t My_id.Map.t
gives you an error
Error: Unbound module Some_type.Diff
you can fix it by writing
type t = (Some_type.t [@diff.atomic]) My_id.Map.t
Applicative functors are not supported by diff in general, so none of the following will work:
type t = Set.M(Int).t [@@deriving diff]
type t = float Map.M(Int).t [@@deriving diff]
type t = Set.Stable.V1.M(Int).t [@@deriving diff]
type t = float Map.Stable.V1.M(Int).t [@@deriving diff]
However, there is specific support for applicative functors for sets and maps: you can use
set
and map
annotations similar to the atomic
annotation.
Here are some examples of what will work:
type t = Set.M(Int).t [@@deriving diff ~how:"set"]
type t = (float Map.M(Int).t [@diff.map]) [@@deriving diff]
type t = { value : Set.Stable.V1.M(Int).t [@diff.set] } [@@deriving diff]
type t = | Value of float Map.Stable.V1.M(Int).t [@diff.map] [@@deriving diff]
The following are not supported by default:
type t = (int, Int.comparator_witness) Set.t [@@deriving diff]
type t = (int, float, Int.comparator_witness) Map.t [@@deriving diff]
But they do work with the set
and map
annotations.
Here are some examples of what will work:
type t = (int, Int.comparator_witness) Set.t [@@deriving diff ~how:"set"]
type t = (int, float, Int.comparator_witness) Map.t [@@deriving diff ~how:"map"]
type t = { value : (int, Int.comparator_witness) Set.t [@diff.set] } [@@deriving diff]
type t =
| Value of (int, float, Int.comparator_witness) Map.t [@diff.map]
[@@deriving diff]
For the set/map annotations to work in general, they need to be able to determine the element type of the set / the key type of the map.
All of the examples so far use a heuristic to determine the key/elt.
By default we assume that the key/elt is X.t
for:
X
as an argument (e.g. Set.M(X).t
, Map.Stable.V1.M(X).t
)X.Set.t
or X.Map.t
X.t
is the first of two parameters for set or first of three for map (so
that it works for (X.t, X.comparator_witness) Set.t
or (X.t, Y.t, X.comparator_witness) Map.t
))If none of the above heuristics apply (or if you want to override the default for whatever reason), you will need to provide the key/elt explicitly.
Any of the following would work:
type t = My_int_set.t [@@deriving diff ~how:"set" ~elt:int]
type t = float My_int_map.t [@@deriving diff ~how:"map" ~key:int]
type t = { value : My_int_set.t [@diff.set (elt : int)] } [@@deriving diff]
type t = | Value of float My_int_map.t [@diff.map (key : int)] [@@deriving diff]
One problem that arises from using atomic
, atomic_using_compare
, set
and map
attributes is that
your type definition may look a bit polluted, since you need to use the attributes in both
the mli and the ml
E.g. the following is rather verbose
module Range : sig
type t =
{ start : Some_type.t [@diff.atomic]
; stop : Some_type.t [@diff.atomic]
}
[@@deriving diff]
end = struct
type t =
{ start : Some_type.t [@diff.atomic]
; stop : Some_type.t [@diff.atomic]
}
[@@deriving diff]
end
If you don't care about exposing what the diff type actually is, you can make the diff abstract.
You can do this either directly by writing:
module Range : sig
type t =
{ start : Some_type.t
; stop : Some_type.t
}
include Diffable.S with type t := t
end
or, equivalently, by using ~how:"abstract"
:
module Range : sig
type t =
{ start : Some_type.t
; stop : Some_type.t
}
[@@deriving diff ~how:"abstract"]
end
Want to use [@@deriving diff]
with stable types and guarantee that [Diff.t]
is also stable?
You can use the [stable_version]
annotation, e.g.
type t = ... [@@deriving diff ~stable_version:1]
The resulting diff type won't change as long as it only references other types that are
both stable themselves and use the diff [stable_version]
annotation (or atomic diffs).
Note that the compiler does NOT check that you got this right, so we suggest adding a bin
digest test for peace of mind.
Only version 1
exists at the moment. There are also currently no plans to change any the
types generated by diff, but the stable_version
annotation will protect you in case we
ever do choose to make changes.
ppx_diff
can be used with Polling_state_rpc
by calling Diffable_polling_state_rpc_response.Polling_state_rpc_response.Make
For example, the following will work:
module Query = Unit
module Response = struct
type t =
{ foo : int
; bar : float
; baz : char
} [@@deriving sexp, bin_io, diff]
end
let polling_state_rpc =
Polling_state_rpc.create
~name:"get-response"
~version:0
~query_equal:Query.equal
~bin_query:Query.bin_t
(module Diffable_polling_state_rpc_response.Polling_state_rpc_response.Make
(Response))