Is it possible to distinguish between absent (no value) and null when serializing?

[GP4cK]

Let's say I have a SimpleValue class with a nullableString:

abstract class SimpleValue implements Built<SimpleValue, SimpleValueBuilder> {
  @BuiltValueSerializer(serializeNulls: true)
  static Serializer<SimpleValue> get serializer => _$simpleValueSerializer;

  String? get nullableString;

  factory SimpleValue([void Function(SimpleValueBuilder) updates]) = _$SimpleValue;
  SimpleValue._();
}

I would like to be able to have this behaviour when serializing to JSON:

final noValue = SimpleValue();
print(serializers.toJson(SimpleValue.serializer, value)); // should print {}

final nullValue = SimpleValue((value) => value.nullableString = null);
print(serializers.toJson(SimpleValue.serializer, nullValue)); // should print { "nullableString": null }

Is it possible?

[davidmorgan]

Please try tagging your Serializer getter with

https://pub.dev/documentation/built_value/latest/built_value/BuiltValueSerializer-class.html

and setting serializeNulls: true, that should do it :)

[GP4cK]

Yes I tried that (it's already in the code I posted above). However it will always print { "nullableString": null }. Instead, I would like that if I don't explicitly set nullableString to null when creating SimpleValue, then the serializer should print {}. Here's a simple repo: https://github.com/GP4cK/built_value_absent

[davidmorgan]

I'm afraid that's not possible; the builder already uses null as the default, explicitly setting to null does not change the state so there is no way to make it serialize differently.

[GP4cK]

Thanks for your answer. Do you think that's a feature that could be developed? I think the package freezed managed to do it. although I haven't looked into the details yet. I'd be keen on helping. Cheers.

[davidmorgan]

Likely not: it would complicate the current implementation for not much benefit.

I'm curious, why do you need this behaviour? Maybe there's another way to achieve what you're trying to do. Thanks.

[GP4cK]

I'm curious, why do you need this behaviour? Maybe there's another way to achieve what you're trying to do. Thanks.

It's because of the issue I linked at the top. I use ferry to generate graphql queries / mutations and ferry uses built_value to serialize the variables of a mutation before sending it to the server.

For example: if I have these graphql objects:

type TodoItem {
  id: ID!
  title: String!
  dueDate: Date
}

# title and dueDate are nullable in the input to only send what you want to update
input TodoInput {
  id: ID!
  title: String
  dueDate: Date
}

If I want to only update the title of a Todo, I can just send the id and the title and leave the dueDate absent. But let's say I want to clear the dueDate, I would need to set the dueDate to null.

If we can't differentiate between null and absent, either:

1. If a field has no value, we serialize it to null. But in that case we could accidentally clear the dueDate
2. If a field is null, we don't send it. In that case, we can't remove a dueDate once it's set

There are some workarounds like sending an array instead of a value and consider that if the array is absent it means no update vs if the array is empty it means you want to set the field to null etc. But I wish there was a more direct way to do things.

[davidmorgan]

Ah, yes, I'm familiar with that kind of usage. Effectively you want another layer of Optional on top of what's already there; you are representing changes to types, rather than the types themselves.

I would like to support that somehow but I haven't figured a way for it to fit in nicely with what we have already. I suspect the 'correct' way might involve a third generated class: in addition to Foo and FooBuilder we'd have FooUpdate which does what you describe. It's not clear to me if updates should be mutable or not; maybe we want FooUpdate and FooUpdateBuilder. Or maybe there is some general approach that avoids having so many new classes.

I suspect this is too big a problem for me to get to any time soon, unfortunately.

[knaeckeKami]

I experimented somewhat successfully with this in ferry_generator, a code gen that generates graphql classes that use built_value.

At the moment I introduce a new Value type. This class just wraps any other value and is used to represent the following states:

• value is present and non-non null ( will serialize to e.g. "key" : "value" in json)
• value is present and null ( e.g. "key" : null )
• value is absent (will not be serialized)

At the moment I generate a custom serializer for any type that has such a Value type, but I'd like to avoid that if possible since I feel this is tricky to get right in all corner cases and built_value has figured that out pretty well in the last 8 years ;).

Do you think it makes sense to add support for this directly in buit_value via the generated serializers?

Or maybe add a Plugin like StandardJsonPlugin that understand such a Value type and wraps/unwraps these values?

(reference: https://github.com/gql-dart/gql/pull/381 )

[davidmorgan]

How does the Value clash distinguish the "present and null" case from the "absent" case, does it have an additional boolean field?

Possibly there could be support for such a boolean field indicating whether the null is present. I'd have to think about it though :)

[knaeckeKami]

At the moment I did it this way:

Value is just a wrapper around a nullable field:

class Value<T extends Object> {
  final T? _value;

  T? get value => _value;

  /// Create a (present) value by wrapping the [value] provided.
  const Value(T? value) : _value = value;

When there is an optional Stringfield named field, it is wrapped in the following way:

Value<String>? get field

So now there are three possible states:

1. field itself is null, field == null
2. field is set to Value(null). field == Value(null)
3. field is set to some non-null String value, like Value("hello world")

So I use the nullability of the wrapper to represent the absent state. However, I just experimented with this yesterday and might come up with another implementation.

It works, but I'm not happy with it in its current form because it breaks the composability of nested builders and it requires custom serializers, so if I ship it, I'll probably come up with something else.

[knaeckeKami]

I have now released a first dev version of this feature in ferry_generator. ferry_generator can now generate built_value classes which support differentiating between null and absent values.

This is done be wrapping each nullable field in a "Value" class

https://github.com/gql-dart/gql/blob/master/codegen/gql_tristate_value/lib/src/value.dart

This is a sealed class with two possible types, PresentValue(value) and AbsentValue().

This allows us to represent three states:

• absent value which should not be serialized const AbsentValue()
• present value which is non-null PresentValue("some value")
• present value which is null PresentValue(null)

in order to make this work, each value - field has to be initialized to const AbsentValue() in the _initializeBuilder. Also, the class needs a custom Serializer which understands to Value type.

An example of Built-Class with value types and serializer can be found here:

https://github.com/gql-dart/gql/blob/master/codegen/end_to_end_test_tristate/lib/variables/__generated__/create_review.var.gql.dart

Is there any interest in adding a feature like this in built_value directly?

[davidmorgan]

Thanks Martin :)

Did I understand correctly that the Serializer of each class with a field of type Value needs modifying currently?

I wonder if you could implement this with a SerializerPlugin:

https://pub.dev/documentation/built_value/latest/serializer/SerializerPlugin-class.html

it gets called during serialization with information about the types, and can modify the data and response. So maybe it can make the changes needed for all types.

[knaeckeKami]

Yes, currently it generates a custom Serializer for every Built class that handles wrapping/unwrapping the Value and serializing only values that are wrapped in a PresentValue() type (if they are optional).

e.g.

    if (_$episodevalue case _i1.PresentValue(value: final _$value)) {
      result.add('episode');
      result.add(serializers.serialize(_$value,
          specifiedType: const FullType(_i2.GEpisode)));
    }

I did really not look into the SerializerPlugin yet, potentially a the custom serializers could be avoided, which I would like.

I'll check it out and see if I can get this working.