search

digital-asset / daml

The Daml smart contract language
https://www.digitalasset.com/developers
Other
802 stars 204 forks source link

JSON decoding / encoding of Java codegen classes #13558

Open TuranDev opened 2 years ago

TuranDev commented 2 years ago

I confirm that, if this is a feature request that has security implications, I already contacted security@digitalasset.com and followed the responsible disclosure policy.

I confirm that this is not a question or a request for technical support by the community, for which the Daml forum is available.

What is the problem you want to solve?

The current template objects are immutable (which is good). Jackson requires an empty constructor (for non final) or a JsonCreator to be specified in order to deserialize.

What is the solution you would propose?

In the confines of how a Template or inner Contract is structured adding the JsonCreator and JsonProperty annotations to the generated Template classes would allow compatibility with Jackson. Another option is perhaps a static fromJson method?

Describe alternatives you've considered

Manually creating mappers for each type to a Jackson compatible type

Additional context

None for now

stefanobaghino-da commented 2 years ago

Are you talking about the Ledger API client classes (like transactions) or the output of the codegen?

TuranDev commented 2 years ago

Output of the codegen

stefanobaghino-da commented 2 years ago

The output of the codegen is supposed to be used to deserialize from the Ledger API, which uses Protobuf as its encoding mechanism.

Can you expand on what is the use case to deserialize from JSON?

TuranDev commented 2 years ago

Mainly around receiving contract or choice arguments from GUI actions and then passing those along to the underlying ledger api's. Ideally without having to manually create mappings for each type of template. Incase you ask why doesn't the UI hit the ledger/json api directly, this is in a world where the notion of a ledger is abstracted away from the end user. Either because they don't care or because one ledger/dap will power multiple derivative GUIs.

Then on the serialization side of things sending results of query's back to the user e.g. a list of "Template.Contract.data" results

stefanobaghino-da commented 2 years ago

Mainly around receiving contract or choice arguments from GUI actions and then passing those along to the underlying ledger api's. Ideally without having to manually create mappings for each type of template. Incase you ask why doesn't the UI hit the ledger/json api directly, this is in a world where the notion of a ledger is abstracted away from the end user. Either because they don't care or because one ledger/dap will power multiple derivative GUIs.

Then on the serialization side of things sending results of query's back to the user e.g. a list of "Template.Contract.data" results

While there may be useful applications for transforming contract data to and from JSON and this feature may be considered in the future, I would still strongly encourage you to take one step further in abstracting away the underlying ledger by not using code that is generated to be consumed by a specific API to leak into the application logic.

S11001001 commented 2 years ago

There are two related ideas here:

  1. support some kind of type-driven JSON encoding for Java codegen output
  2. support the Daml-LF JSON encoding format for Java codegen output

(2) is one instance of (1), but there are many other incompatible encodings that satisfy (1). Since our supplying any of these would be incredibly confusing, if we implement any JSON encoding for Java codegen values, it ought to be (2).

Improving support for Jackson is a possible path to (1), but probably not how we would go about supporting (2). There are too many possibilities for subtle incompatibilities. So, broadly speaking, even if we were to support JSON, I do not think we should provide support for Jackson in the ways suggested in this issue.

If we were to support Java-codegen/JSON codecs, I suggest we implement (2) by using com.daml.ledger.javaapi.data.Value as a pivot type, and some runtime representation of the Daml type associated with each codegen type, because encoding to JSON does not require type info (V is Value here):

https://github.com/digital-asset/daml/blob/281de9ac2e2989ef73de82bf39078755f310a9ce/ledger-service/lf-value-json/src/main/scala/com/digitalasset/daml/lf/value/json/ApiCodecCompressed.scala#L42

but decoding from JSON to Value does:

https://github.com/digital-asset/daml/blob/281de9ac2e2989ef73de82bf39078755f310a9ce/ledger-service/lf-value-json/src/main/scala/com/digitalasset/daml/lf/value/json/ApiCodecCompressed.scala#L297-L302

(You need some equivalent of both iface.Type and Ref.Identifier => Option[iface.DefDataType.FWT] to make this work, hence the requirement for Daml type metadata.)

This would mean that only Java codegen users who truly need JSON would need to use the lf-value-json and daml-lf/interface libraries and everything those imply.

If your heart was then truly set on using Jackson, I suspect you could register a codec with JSON that would then use these libraries as support.

As @stefanobaghino-da mentions, however, we generally advise against using Daml types directly as a domain model in client applications, or attempting to fit Daml type definitions directly to an external domain model, as also discussed over on the forum. In the Daml SDK, we would be most interested in the above-outlined feature as a way to interact with ledgers in new ways; the feature would satisfy the use of Daml types as JSON-encodable client domain model only incidentally.

S11001001 commented 1 year ago

Once you have all of these, you can compose them to create an encoder javaapi DefinedDataType -> javaapi Value -> lf.value.Value -> JSON.

You can also create a decoder, but it's a little trickier: (ContractTypeCompanion, type model) -> typesig type model, DamlLfTypeLookup -> (JSON -> javaapi Value -> codegen value)`

S11001001 commented 1 year ago

@chunlokling-da

Are "codegen a model of types" and "new Java bindings representation of type" referring to the same thing?

No.

So just like javaapi.data.Value is just a Java copy of lf.value.Value, with Java idioms and whatnot, the latter is doing that. So the discussion is whether to do that.

And if you do that, then the former is to generate code that builds structures of that new representation. If you don’t do it, then you still need to codegen a model of types, it’s just that the target is the existing model from api-type-signature that can be used directly when calling the ApiCodecCompressed functions.

chunlokling-da commented 1 year ago

Based on the comment by @S11001001 . Created sub issues:

Json Encoding:

Json Decoding:

chunlokling-da commented 1 year ago

@S11001001 :

And if you do that, then the former is to generate code that builds structures of that new representation. If you don’t do it, then you still need to codegen a model of types, it’s just that the target is the existing model from api-type-signature that can be used directly when calling the ApiCodecCompressed functions.

might be worth a bigger discussion specifically to decide whether to generate api-type-signature type model directly in Java codegen or write a new type model in Java to codegen

We have decided to depend on the scala typesig lib directly instead of creating java version of it for codegen