EJSON - fromEJSON does not respect fields that have default value.

[Rob-Biemans]

What happened?

Because MongoDB/Atlas Search does not transfer information about fields that have their default values (nullable). The fromEJSON throws InvalidEJson because the fields are not set with NULL.

Repro steps

Create a model with nullable fields like:

  class _AddressModel {
    late String? type;
    late String street;
    late String? suburb;
    late String city;
    late String? state;
    late String? postalCode;
    late String countryCode;
    late _LocationModel? location;
  }

Try to convert Map (JSON) to Object Note: type, suburb, state and postalCode have their default value (null) so nothing is returned from Atlas Search

AddressModel testWithCrash = AddressModel.fromEJson<AddressModel>(
  {
    countryCode: NLD, 
    street: Malieveld, 
    city: Den Haag, 
    location: {
     coord: [{$numberDouble: 52.085413}, {$numberDouble: 4.317371}],
      type: Point}
  }
)

  static AddressModel _fromEJson(EJsonValue ejson) {
    return switch (ejson) {
      {
        'type': EJsonValue type, <--------- Throws because the json above does not contain that field.
        'street': EJsonValue street,
        'suburb': EJsonValue suburb,
        'city': EJsonValue city,
        'state': EJsonValue state,
        'postalCode': EJsonValue postalCode,
        'countryCode': EJsonValue countryCode,
        'location': EJsonValue location,
      } =>
        AddressModel(
          type: fromEJson(type),
          street: fromEJson(street),
          suburb: fromEJson(suburb),
          city: fromEJson(city),
          state: fromEJson(state),
          postalCode: fromEJson(postalCode),
          countryCode: fromEJson(countryCode),
          location: fromEJson(location),
        ),
      _ => raiseInvalidEJson(ejson),
    };
  }

Version

Flutter 3.19.3 • channel stable - Dart SDK version: 3.3.1 (stable)

What Atlas Services are you using?

Both Atlas Device Sync and Atlas App Services

What type of application is this?

Flutter Application

Client OS and version

realm: ^3.0.0

Code snippets

No response

Stacktrace of the exception/crash you're getting

Exception has occurred.
InvalidEJson (Invalid EJson for AddressModel: {countryCode: NLD, street: Malieveld, city: Den Haag, location: {coord: [{$numberDouble: 52.085413}, {$numberDouble: 4.317371}], type: Point}})

Relevant log output

No response

[sync-by-unito[bot]]

➤ PM Bot commented:

Jira ticket: RDART-1062

[Rob-Biemans]

Currently creating a hacky way so I don't keep stuck at my work; I just want to add that setting the field "type" as null when undefined is OK after the fact.

{ 'type': null, }

But it gets little complicated because when the schemaProperty has a collectionType of either List/Map/Set. It also throws like: Note: Posts is a RealmList"PostModel"

{ 'posts': null <-- Crash, because it is supposed to be [] instead due schemaProperty.collectionType.name == List }

[nielsenko]

@Rob-Biemans The advantage of requiring all fields in the switch (even nullables) is that it gives a firmer hold on the shape of the object. If we allow nullables to be absent, then deserialization becomes more ambiguous. Say both a Person and a Dog has a name, but the later has an optional owner, then we can disambiguate on the owner field.

Often we know the type to deserialize to, but for RealmValue we do not know the inner type ahead of time. In this case we will deserialize to the first matching type, and if both Person and Dog match, it is undefined which it will be.

Notice this an issue even today, if two different realm objects has identical properties, but a lot less likely to happen.

Anyway, you have me convinced that we need to support this. At least for realm objects.