Proposal - Object Validation

[AnthonyMDev]

Goals

Support validation of edited objects during commitWriteTransaction.

Functionality

An Object may override an validate() function to throw an Error if the object is not valid to be written to the Realm. During commitWriteTransaction if any edited objects throw an Error, the write will fail and the user may handle the error.

Implementation

A ivar editedObjects of type NSSet<RLMObjectBase *> will be added to RLMRealm. When an object has a value set during a write transaction, it will be added to the editedObjects in RLMDynamicSet. In commitWriteTransactionWithoutNotifying:error: the edited objects will all be validated and if any of them fail, an error will be thrown. The editedObjects  set would then be cleared out before another write transaction began.

Concerns/Questions

• This implementation may impact performance. I anticipate that the performance hit will be very small though.

• If the validation fails, how is that handled? Do we revert the entire write transaction?

[TimOliver]

Hi there @AnthonyMDev!

Thanks for the feature proposal! That's a very interesting idea!

I'm a little curious though, what sort of use-cases do you have where you'd need to have this functionality in Realm?

Thanks!

[jpsim]

Thanks for your interest in contributing to Realm and for clearly putting in some careful thought into how this should be done.

These are interesting ideas that will need further consideration, but I appreciate that you're sharing your thought process in detail prior to diving in with a lot of work.

The Realm team might not be super responsive over the holidays to work with you on this, but here are some things to consider:

1. How can you make sure that this validation is applied even when Objective-C/Swift accessors aren't created? For example, when using KVC on Realm Collections, Objective-C/Swift classes are never created for the individual Realm objects whose properties are being modified.
2. How should the concept of validation be enforced when not all changes to a Realm are being made through the Objective-C or Swift SDK? For example, if changes are made by the Realm Object Server or by other SDKs via synchronization?

One thing that I'm almost certain of is that the only way for this to work would be to prevent a change that doesn't pass validation from being committed to the Realm, since rolling back partial commits isn't supported, and doing so at a higher level is sure to lead to very complex, if not impossible, behaviors with synchronization, where the fact that operations need to be reorderable while preserving the same deterministic merge result is a necessary constraint that must be upheld in any operational-transform scenario.

I'll let you sit on this for a bit, and I hope we can resume this conversation probably after the holidays.

[AnthonyMDev]

Thanks for the positive words guys!

@jpsim I'll consider the points you've made and do some research. We can get back to this after the holidays.

@TimOliver I have already got multiple use-cases for this in my own applications. I'll outline some of them for you tomorrow.

[AnthonyMDev]

Alright, I've done a little bit of thinking. I'm not 100% familiar with all of the inner workings of Realm just yet, but I see the problems that you've brought up and I have some ideas to start discussing.

How can you make sure that this validation is applied even when Objective-C/Swift accessors aren't created? For example, when using KVC on Realm Collections, Objective-C/Swift classes are never created for the individual Realm objects whose properties are being modified.

What if we made the validation method a class method on RLMObject. It would take an instance of Self as a parameter and validate that object. Then, anywhere where we are saving/creating an object with KVC we can still call the class method.

Alternatively, we could create some sort of pre-defined property validations and use some sort of annotations for them. If I'm understanding correctly, we could then map the validation on to the RLMProperty objects. I haven't done enough work looking at how RLMProperty objects are derived or utilized yet, so this isn't fully fleshed out, but I am going to look into it more. This is what I imagine it might look like if we go down this route:

class func propertyValidation() -> [String: [PropertyValidator]] {
    return [ 
            "name":        [.required] // or "nonnull"
            "dateCreated": [.before(NSDate())],
            "age":         [.required, .between(1, 99)]
            "followers":   [.countBetween(1, 10)]
            ]
}

How should the concept of validation be enforced when not all changes to a Realm are being made through the Objective-C or Swift SDK? For example, if changes are made by the Realm Object Server or by other SDKs via synchronization?

I don't know a lot about this yet, as I'm not using the Realm Object Server personally. I can look into it a bit, but I'd love input from the team. If the validation was a part of the schema, rather than a custom validation function, would that be able to be synced over to the Realm Object Serveras well and handled then? This seems like it would likely be possible if we implement the second proposed solution with the pre-defined validation.

This implementation would probably go all the way back to the realm-object-store repository. Which would be nice because it means that the implementation for the Java SDK wouldn't be too difficult at that point.

@jpsim Hope to hear your thoughts soon!

[AnthonyMDev]

@TimOliver From my example above, you may be able to derive some basic use cases for this.

My primary use case personally is for required fields. While RealmSwift supports optional or required primitive value properties, because of the introspection in Swift we can't make required relationship properties. The .required property validator would be very useful to me.

Relationships

If I validate a relationship property as .required, I could then implicitly unwrap the relationship and feel pretty safe about it.

dynamic var owner: User!

Now I don't need to unwrap my property with if let statements all over my application.

String

Minimum length requirements on String properties can help make my code cleaner because I don't have to check if my String is empty every time I want to use it.

class func propertyValidation() -> [String: [PropertyValidator]] {
    return [ 
            "optionalNonEmptyString": [.min(1)]
            "requiredNonEmptyString": [.min(1)]
           ]
}

dynamic var optionalString: String? // May be `nil` or empty.

dynamic var requiredString: String // May never be neither `nil` but may be empty.

dynamic var optionalNonEmptyString: String? // May be `nil`, but never empty.

dynamic var requiredNonEmptyString: String // May never be neither `nil`  nor empty.

Now see how I use each of these when displaying my model object. In this example, I am displaying a UILabel with the string value, but if the string is either nil or empty, I need to hide the label. (For example, when using a UIStackView so that the label doesn't create empty space.

Optional With No Validation

guard let text = object.string else {
    label.hidden = true
    return
}
label.text = text
label.hidden = text.isEmpty

Required With No Validation

label.text = object.string
label.hidden = object.string.isEmpty

Optional Non-Empty

label.text = object.string
label.hidden = object.string != nil

Required Non-Empty

label.text = object.string

You can see that the versions in which I can already assume that the string is not empty are more concise. Over a large code base, checks for empty strings can add up to a lot of extra lines.

Numbers

This same value added from the String example can be said for Int, Double or Float properties in which 0 (or any default value) would be used to signify a value that hasn't been set. RealmOptional works fine when you want to allow the value to actually be nil. However in the case where the property is required, but isn't set properly upon insertion, this will now throw an error, alerting you to the issue.

Importing JSON

The most common use case for me personally is in validating that JSON data sets up my objects correctly. I initialize my object, map my JSON on to it, and then save the object. But what if I have a required String property and received invalid JSON that doesn't include the key for that property?

Currently, my object would save with the default value for the property (usually an empty string).

I hope that I have convinced you that my proposal would be a valuable addition to Realm.

Thanks for your time.

[AnthonyMDev]

@jpsim Hope you had a great holiday! Looking forward to your comments on my proposal so I can begin moving forward!

[m1entus]

+1