google / protobuf.dart

Runtime library for Dart protobufs
https://pub.dev/packages/protobuf
BSD 3-Clause "New" or "Revised" License
527 stars 183 forks source link

Support for EmitDefaults during JSON Serialization via toProto3Json() #585

Open jmartin127 opened 2 years ago

jmartin127 commented 2 years ago

The proto3 spec states:

A proto3 JSON implementation may provide the following options:

Emit fields with default values: Fields with default values are omitted by default in proto3 JSON output. An implementation may provide an option to override this behavior and output fields with their default values.

An example of this for the protoc-gen-go plugin is EmitDefaults option: https://pkg.go.dev/github.com/golang/protobuf/jsonpb#Marshaler

I think it would be great if we could achieve this same functionality via the protoc-gen-dart plugin.

As far as I can tell, this functionality is not currently supported by this plugin. Please confirm if that is the case. If this functionality is not currently supported, I would like to look into adding it, as I think it could be a useful feature for other uses of the plugin as well.

Steps to reproduce:

  1. Create a proto file:
    
    syntax = "proto3";

message Money { string currency_code = 1; int64 units = 2; int32 nanos = 3; }

2. Run protoc with the `dart_out` option to run the `protoc-gen-dart` plugin
`protoc --dart_out=lib/src/generated -Iprotos protos/money.proto`
3. Construct a message with the `units` field populated:

final Money amount = Money.create() ..currencyCode = "USD" ..units = $fixnum.Int64.parseInt("1") ..nanos = 750000000; var json = jsonEncode(amount.toProto3Json()); print('Serialized JSON: $json\n');

4. Observe that the serialized JSON contains this field:
`{"currencyCode":"USD","units":"1","nanos":750000000}`
5. Construct a message without the units field:

final Money amount = Money.create() ..currencyCode = "USD" ..nanos = 750000000;


6. Observe that the serialized JSON does not contain this field:
`{"currencyCode":"USD","nanos":750000000}`

For our use case, this is problematic, because we'd like to store the serialized JSON into a `jsonb` field in Postgres, and with so many potentially missing fields, they jsonb queries can get very cumbersome.

Again, we're willing to contribute to the project to provide serialization of fields with default values as an option, but want to double check this isn't already available in the plugin, and just hadn't figured out how to do it.
jmartin127 commented 2 years ago

Note: While testing a solution for this, I found that this package actually doesn't follow the protobuf spec. The spec states that: Fields with default values are omitted by default in proto3 JSON output. It also states that For strings, the default value is the empty string. This package, on the other hand will emit the JSON key as long as the field was set.

Example 1 (currencyCode is set to 'USD', serialized JSON contains this key as expected):

  final Money amount = Money.create()
    ..currencyCode = 'USD' // String type
    ..units = $fixnum.Int64.parseInt("1")
    ..nanos = 750000000;

Output 1:

Serialized JSON: {"currencyCode":"USD","units":"1","nanos":750000000}

Example 2 (currencyCode is set to the Default value '', serialized JSON should not emit this key according to the spec, but it does):

  final Money amount = Money.create()
    ..currencyCode = '' // String type
    ..units = $fixnum.Int64.parseInt("1")
    ..nanos = 750000000;

Output 2:

Serialized JSON: {"currencyCode":"","units":"1","nanos":750000000}

Example 3 (currencyCode is never set, serialized JSON is as expected, and does not emit this key):

  final Money amount = Money.create()
    ..units = $fixnum.Int64.parseInt("1")
    ..nanos = 750000000;

Output 3:

Serialized JSON: {"units":"1","nanos":750000000}