How can we keep generated PKL go structs in sync

[ycz-dd]

I'm experimenting with PKL now as a source of truth for a schema (like JSON schema). (sidebar: Is this a good use for PKL?)

Here's an simple "schema" or "spec" example:

// response.pkl
Headers: Mapping<String, String>
Body: Mapping<String, String>

So afterwards, I've generated go structs from pkl-gen-go and I want to ensure that generated go code is in sync with the PKL files.

My approach is using unit tests that'll break so unsync'd PKL <> Go code won't get merged. I'm following off of the pattern here:

// unit test table
// ...
         for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            evaluator, err := pkl.NewEvaluator(context.Background(), pkl.PreconfiguredOptions)
            if err != nil {
                panic(err)
            }
            defer evaluator.Close()
            if err = evaluator.EvaluateModule(context.Background(), pkl.FileSource(tt.pklPath), tt.expectedStruct); err != nil {
                panic(err)
            }
            assert.Nil(err)
        })
    }

That seems promising for PKL files that are already filled out, not schemas. So it seems to break in basic types like String and Int.

To take a step back,

1. is using PKLs to document a schema (and generating source files from said schema to work with) a good idea?
2. if so, is using unit tests to read in the schema PKL file with the generated struct a good idea to keep codegen'd files in sync?

[ycz-dd]

Setting some 0 values like will make this unit test to sync generated Go structs with pkl sources work. I feel pretty iffy about that for a schema

// request.pkl
status_code: Int = 200

[bioball]

So you're only using Pkl for schema, and not data?

A simple test would be to run pkl-gen-go within CI, and fail your CI job if there is diff. In this approach, I'm thinking you'd also check the generated structs into your repo.

You can certainly use Pkl to describe schemas. But note that Pkl's types are more expressive than Go's types, and certain things get lost. For example: Int(isBetween(5, 10)) simply turns into Go's int.

By the way, you can also generate OpenAPI schemas from Pkl classes, or simply use Pkl to describe JSON Schema or OpenAPI Schema directly.

[ycz-dd]

Thank you Daniel!

Pkl as a schema

I want to use Pkl as a spec or a schema and codegen Go structs from that (maybe explore OpenAPI or codegen on Kotlin/Swift). Great call on the expressiveness difference between Go and Pkl.

How do we ensure generated go structs are in sync with that Pkl schema?

So you do not recommend using

evaluator.EvaluateModule(context.Background(), pkl.FileSource(tt.pklPath), tt.expectedStruct)

to verify specs.

Lemee see if I understand your suggestion. 😭 I can call pkl-go/cmd/pkl-gen-go in a unit test right? And fail on any file change. I see that will at least update the visited timestamp. I'll try that and get back to you. Thank you Daniel!

What is evaluator.EvaluateModule supposed to be used for?

So evaluator.EvaluateModule(..) is more about inflating configuation files rather than validating specs. I don't like setting default or zero values for a spec.

evaluator.EvaluateModule(..) is more for loading filled out Pkl files. What I got from the tutorials is: Pkl is super nice as a typed YAML/JSON for configuration. I see less tutorials for Pkl as a schema 🤔

[ycz-dd]

Thank you Daniel, that approach works. There's some quirks with the codegen (specifically #50 ) that hopefully is more configurable, but using Pkl as a schema is nice!

[eli-l]

It would be great we have a validation possibility on the Go side, so we can ensure we loaded proper config (it might be 100% correct from Pkl side but we might load it into wrong Go struct).

The problem is rather on the Go side, not Pkl itself. What I've being suggesting is to implement a similar approach as it happens in other code generators, where optional values are transpiled to pointers (nil-able) and mandatary properties are just types (passed by value). This would at least save us from panics accessing properties on nil and make our Go structs more reliable.

I see here @ycz-dd is referring to the same problem and with a bit different perspective, but to make it a reliable tool we definitely must take care of ensuring our structs are correct. Validating each field for (!= nil) is a nightmare and makes completely no sense to use Pkl bindings for go.

I'm glad someone else is attending this repo and asking important questions, would be great to hear back from you.