beacon-biosignals / Legolas.jl

Tables.jl-friendly mechanisms for constructing, reading, writing, and validating Arrow tables against extensible, versioned, user-specified schemas.
Other
24 stars 2 forks source link

add convenience macro for declaring a new version of a schema which differs only from an older version by additional fields #121

Open ericphanson opened 2 months ago

ericphanson commented 2 months ago

e.g.

@schema "foo.bar" Foo

@version FooV1 begin
   a::Int
end

@incremental_version FooV2 begin
    FooV1...
    b::Float64
end

could expand to adding the a::Int field into FooV2.

Note that this differs from FooV2 > FooV1 (which is not allowed) bc that affects the identifier (i.e. the metadata in the table, which means FooV1 must always exist when FooV2 does and it semantically inherits from it), whereas this has no semantics on the level of the table itself and is simply a convenient way to declare the fields for FooV2. With this macro, one could later delete FooV1, replace @incremental_version with @version, and copy-paste the fields from FooV1 into the definition of FooV2, without breaking anything that uses FooV2.

Note I'm not sure this is the best syntax (FooV1...); that also opens the door to multiple schema-versions and field-ordering choices that maybe we don't want to expose. It could be @additive_version FooV2 ... which introspects the "V2" literal to decide FooV1 is being added to, or something like that (without the FooV1... bit in the body).