Apply interfaces to generated classes: Templates, Data Objects, Choices + CreatedEvent Core contract data

[StephenOTT]

What is the problem you want to solve?

When working with the generated java classes, there are some QOL aspects that are painful / create lots of work on the dev side to build boilerplate code:

1. It would be very helpful to have interfaces applied to the generated templates: Templates have Template.class, but there is nothing really useful with that interface. It would be nice if each template generated its own interface so we could easily handle data transformation objects. If a template has 3 fields, and you want to apply some annotations to those fields, it would need to modify codegen, modify the generated classes, or manage the list of fields in your own classes. If there was generated interfaces then we could apply this to our own classes and easily keep track of missing/changed template fields.2.

2. Data objects / Records: https://docs.daml.com/daml/intro/3_Data.html#records get generated as classes, but have no interface or identifiers. There should be an interface to identify this is a Record and should be an interface we can implement on our own classes (similar to item 1)

3. Choice classes have the same issue as item 1 and 2: there are no interfaces to identify it as a choice with the fields associated with the choice and lack the interfaces for implementation on our own classes.

4. finally CreatedEvent has many fields that are "core contract" fields such as observers, signatories, contract id, etc. But there does not seem to be any core interface that provides these fields that we can implement on our own DTO classes. So the dev has to re-build all of these structures and then manage them.

Overall these feature request is about requesting to provide QOL interfaces so devs can have compile time information about the typical business data being used and allow other tooling to leverage that informations. For example having proper hierarchy of interfaces applied could allow DI tooling to generate more classes at compile time to create Repository implementations, etc.

Of course we can do this after generating the classes, but it becomes work that needs to be re-done on each version/re-generation.

[stefanobaghino-da]

I think there are way too many ideas in here and some of them are likely already tracked in our issues. @S11001001 @ray-roestenburg-da Can you have a look here, please?

[StephenOTT]

@stefanobaghino-da the overall issue is lack of interfaces being applied to the generated classes. Each of my examples are just usecases on the same underlying topic.

If you create a label or keyword to group these category, I would be happy to create the missing specific issues.

[StephenOTT]

Another example:

The Tuple classes that are generated (often because of their common use in Template Keys). They have no interfaces so becomes difficult to build generic transformers around: "If it is a DA Tuple then convert into X format"

Keys become this: It could be a String, it could be a Tuple (with multiple variants) and it could be nested tuples as per https://discuss.daml.com/t/input-limit-on-template-key/3029/3?u=stephenott and it could be a Record object.

[S11001001]

2. There should be an interface to identify this is a Record and should be an interface we can implement on our own classes

13766, but the further idea of "implement[ing] on [your] own classes" is subject to the caveat below.

4. CreatedEvent has many fields that are "core contract" fields such as observers, signatories, contract id, etc. But there does not seem to be any core interface that provides these fields that we can implement on our own DTO classes

As a rule, the goal of our codegen and the bindings model library is to hold data, not data-with-behavior. So CreatedEvent is the product of its element fields, no more, no less.

This applies to (1) and (3) as well. Consider that a choice argument is just a record. It, by itself, does not "do" anything; it doesn't have behavior. So what would be of interest to implement in custom classes?

Extensions to codegen, in general, take the form of making the static dataflow more convenient to use. The most complex form of this in the current issues is #14312. A recent example was #13724.

The Tuple classes that are generated

The above applies to this as well. Tuples are the product of their elements, no more, no less.

[StephenOTT]

@S11001001 the codegen creates the data classes, but those data classes have variation between them with common parents and no way to extend or implement:

Examples:

1. CreatedEvent is Final and has no interface. So we end up having to build DTOs manually without any compilation support to detect differences or missing fields.
2. Templates have a Template interface which is a start, but there is no interface for the fields of that template. Yes it is data, but It will be very easy to miss changes when re-building the template from codegen due to changes.
3. Building on item 2: we then end up with different data storage needs: We connected all of the templates into Infinispan / Protostream and so the fields needed @ProtoField and @ProtoFactory annotations. We could also manage this in ~weird out-of-band formats, But it is much easier and type safe (+ compilation checks) to have an interface that we can easily implement on a data class and apply the annotations there. If we re-run codegen we don't lose the annotation placement.
4. CreatedEvent's key field uses Optional but the Value interface is not applied on Tuple2 ( i am not saying it should, just showing the work involved), and so additional type checking is required when doing handling on how to manage the key value for downstream transfer or storage.

[StephenOTT]

This applies to (1) and (3) as well. Consider that a choice argument is just a record. It, by itself, does not "do" anything; it doesn't have behavior. So what would be of interest to implement in custom classes?

@S11001001 But is it still a "Choice". As an example, if Choices have const data about what Templates the choice belonged to, and proper interface to identify it was a choice, I can use that data for compile time code generation of endpoints, Ui components, data config helpers, etc.

[S11001001]

the codegen creates the data classes, but those data classes have variation between them with common parents and no way to extend or implement

The output of the codegen is intended for interaction with the ledger API; it is not meant to be the basis of a data model for the remainder of an application.

If you wish to nevertheless do metaprogramming on its output, I suggest using metaprogramming tools that can do transformations of POJOs rather than requiring inline annotations. I know more about the Scala ecosystem, so I'll have to provide an example in that space, but: Shapeless allows you to build generic transformations on arbitrary case classes and discriminated unions thereof, without having to change those classes. So users can use it on the Scala codegen output without it making any references to Shapeless whatsoever.

[StephenOTT]

The output of the codegen is intended for interaction with the ledger API; it is not meant to be the basis of a data model for the remainder of an application.

:) I think we are saying the same thing then! :). I agree that the output of the codegen is for interaction with the API. So we should have the interfaces needed to create DTOs to convert from the codegen data classes into our own classes. Without the interfaces we have to do this manually. With the interfaces it is: "implement interface, IDE: Add constructor and implement methods". When codegen is re-run on that project (maybe because of a upgrade) then it would be easy to compile time check for changes in the DTOs (as the interfaces would pick up changes, missing impl, etc), otherwise it is back to Git Diff comparisons..

(I would prefer not to apply the annotations on the Template classes. But I would have to manually re-implement every data field because there is no interface for each template).

To build on this further: If template classes did not have the Template interface, how would you know the difference between a Choice class and a Template Class?