search

twitter / finagle

A fault tolerant, protocol-agnostic RPC system
https://twitter.github.io/finagle
Apache License 2.0
8.79k stars 1.46k forks source link

'passthrough_fields' gets added to json serialisation when handling generated Thrift objects #776

Closed cosmir17 closed 5 years ago

cosmir17 commented 5 years ago

'passthrough_fields' gets added to json serialisation when handling generated Thrift objects

Expected behavior

I would like to remove 'passthrough_fields' field from json body.

Actual behavior

'passthrough_fields' gets added to json body

Steps to reproduce the behavior

I am using json4s-jackson library and DefaultFormats class. https://mvnrepository.com/artifact/org.json4s/json4s-jackson

implicit val playableItemFormats = new DefaultFormats {
    override val emptyValueStrategy = EmptyValueStrategy.preserve
  }

Would it appear if we use a different json serialisation library?

'scrooge' generates 'passthrough' field when generating classes from Thrift schema (https://github.com/twitter/scrooge). I don't mind it gets added to generated object but I don't want to include this field in my json body. I suppose Finagle may have it's own serialiser for json. Or Should I post this issue on 'scrooge' github issue page?

enbnt commented 5 years ago

Hi @cosmir17,

The pattern we typically follow at Twitter is to separate the RPC data/serialization layer (the Scrooge generated code) from the models that represent our domain data. We recommend this internally, not only for JSON data, but for classes that your business logic interact with as well. There are limitations to what the Thrift IDL can represent, which creates some hurdles, and we don't mind paying the penalty of converting the data to make it easier to work with.

We recommend using case classes to represent the JSON schema, where we also have finer control over what gets serialized or deserialized to the JSON structure. We also use Jackson within our frameworks to deal with JSON serialization and deserialization (and don't have as much familiarity with json4s). In Jackson you have the ability to register custom serialization and deserialization modules that would give you control over whether or not to include a field in the serialized output. It might be worth looking to the json4s community to see if there is a similar method.

I also noticed that you reached out in the Finatra Gitter and were chatting with @cacoco - is this still an issue for you? Or are you comfortable closing this out? Thank you.

mosesn commented 5 years ago

As @enbnt said, we don't recommend that you use scrooge-generated structs to serialize to json using json4s-jackson. I'm going to close this issue for now.