ipld / go-ipld-prime

Golang interfaces for the IPLD Data Model, with core Codecs included, IPLD Schemas support, and some handy functional transforms tools.
MIT License
132 stars 50 forks source link

Incremental schema type building #568

Closed hannahhoward closed 2 months ago

hannahhoward commented 2 months ago

What

Enable various methods for building schema type systems more incrementally.

Why

Right now, if I have two seperate schema DSL text files, and I want to combine them into one type system, I have to concatenate the bytes for each file and then use IPLD.LoadSchemaBytes()

This is presumably quite slow as it involves a full parse to DMT form followed by a compilation.

Similarly, if I have a schema.Type that doesn't come from a DSL, for example one I received by using bindnode.Wrap's internal inferSchema, I can't combine into DSL schema that references the type

You can generally imagine this as a schema form problem of a go generic. i.e.

type MyStruct[T any] struct {
   A string
   B *int
   C T

mapping to

type MyStruct struct {
   A String
   B optional Integer
   C T
}

with a second file defining T for a specific case:

type T struct {
   D String
   E String
}

How

It seems like there are a few changes one could make.

schemadmt.Parse currently adds basic types, and calls ValidateGraph. One could a expose a slightly more low level API that doesn't do all this (instead just accumlating types from a schema DMT)

another option would be to enable merging two type systems. In such a case, duplicates would be accepted as long as the kinds matched. this would be great for being to merge one complex schema.Type into another type system.