Consider making initWithValue: a deep copy of the value

[balazsgerlei]

Goals

The current Realm API seemed to be focused on accessing persisted object "on-the-fly" from anywhere in the app. While this can be beneficial and simple to use, a more complex, layered architecture cannot be achieved with this. I'm talking about an architecture where database access is limited to a separate Data layer so effectivelyI would only use Realm there. I'm trying to implement this architecture using RxSwift, where I would like to pass business entities (which are Realm objects) via Observables to the Domain layer, where my business logic is implemented using RxSwift operators and finally, the result will arrive to the Presentation layer which includes ViewControllers and Views. The core of this is that everything before the Presentation layer have to be done in a separate thread asynchronously which could be achieved easily with RxSwift. Realm objects however cannot be passed between threads, so it makes this approach impossible. However, in Java, Realm already has a solution which would be nice in Swift. More details: https://realm.io/news/realm-java-0.87.0/ https://github.com/realm/realm-java/tree/master/examples/rxJavaExample

Expected Results

Add copyFromRealm()method to the Swift API of Realm to enable passing a "detached", in-memory version of Realm objects between methods, thus enabling using RxSwift in a multi-layered architecture with business logic in a separate domain layer and a different thread (RxSwift scheduler).

[jpsim]

Why not use Object.init(value:) passing in a persisted object?

[D-Mx]

Object(value: ) detaches the object, but only if it doesn't have nested data structures. Trying to thread-access/modify a related object from a to many relation still throws IncorrectThreadException or Attempting to modify object outside of a write transaction.

[balazsgerlei]

Of course an even better solution would be thread handoff of Realm objects, like in this RxJava issue: https://github.com/realm/realm-java/issues/1208

But I understand that it is a new and complex feature, and until then, a copyFromRealm method like in RxJava would be sufficient. And yes, it would be only helpful if it contains nested data also (like in Realm for Android).

Javadoc for the mentioned method: https://realm.io/docs/java/0.87.0/api/io/realm/Realm.html#copyFromRealm-E-

[jpsim]

Of course an even better solution would be thread handoff of Realm objects

Asynchronous transactions is being tracked as #3136.

But I understand that it is a new and complex feature, and until then, a copyFromRealm method like in RxJava would be sufficient. And yes, it would be only helpful if it contains nested data also (like in RxJava).

We considered this when designing initWithValue: but opted against it because of the potentially serious ramifications of realizing a large object graph into memory.

[balazsgerlei]

You may missunderstood me. I don't want to have asynchronous transactions. I also don't know how familiar are you with Reactive Extensions. What I would like to do is to have a class which access the locally saved data in Realm and return an Observable. I would like to do this all in a background thread. Than another class, which subscribes to this observable, does some required transformation on the data which are emitted by this observable. This data is of course a realm object or a list of realm objects. Than, I switch threads to the main thread, and display this data in the UI. This is fairly straightforward with RxSwift ("subscribeOn" and observeOn" operators and schedulers) but a problem is that as soon as I try to modify the Realm object from the background thread, it throws an exception as it is not allowed to pass a Realm object between threads.

The current solutions I know about are:

1. Only pass along id's and re-query the Realm object after switching threads. But this eliminates the advantages of layered architecture, as one of the aim is to abstract away any databse operations from the UI layer.
2. Create another instance of the queried object which is not attached to Realm. Then manually copy all the properties and contained data into it, all before switching threads. And than do this again when I modify an object.

Realm for Java had this exact same issue but recently they introduced the operator I talked about: copyFromRealm. In the https://realm.io/news/realm-java-0.87.0/ I linked there is an example (in Java):

// Get different versions of a person
realm.where(Person.class).findFirst().<Person>asObservable()
        .map(new Func1<Person, Person>() {
            @Override
            public Person call(Person person) {
                // Convert live object to a static copy
                return realm.copyFromRealm(person);
            }
        })
        .buffer(2)
        .subscribe(new Action1<List<Person>>() {
            @Override
            public void call(List<Person> persons) {
                // Without `map` and `copyFromRealm`, v1 and v2 would be the same
                Person v1 = persons.get(0);
                Person v2 = persons.get(1);
            }
        });

Why is memory a problem when it is not in Java? Of course there is a depth limit to conatined data in copyFromRealm implementation in Java. Also, I thought that similar API is an aim with the multiplatform development of Realm (this is one of the reason I choose Realm as I develop a multiplatform app in Java and Swift).

Or maybe do you have any other suggestion to my problem?

[AliSoftware]

I've a similar request as we've seen inconsistent (even dangerous) behavior because we lack this ability to deep-copy an object graph in RealmCocoa like we can do in Java using copyFromRealm.

(dangerous because one can then modify the persisted Realm… even from a detached, standalone object, simply by traversing a relationship)

@jpsim typical use case if that helps you understand the need for this:

let standalonePerson = Person(value: aPersonFetchedFromTheRealm)
realm.write {
  standalonePerson.dog.name = "Rex"
}

standalonePerson.realm is nil once detached from the Realm, but standalonePerson.dog isn't detached, so the dog gets modified in the DB even if the Person is detached from the DB

---

Extract of a discussion with @bdash on Slack on the subject:

[bdash] Java’s copyFromRealm() is an eager deep copy. Cocoa’s init(value:) is a shallow copy. … [bdash] Yes, I believe standalonePerson.dog will still be a persisted object and thus will still be modified. [bdash] I don’t think init(value:) was ever intended as a copy operation, but as a way to initialize an object from a dictionary or array like you’d get from JSON. However, that does mean we’re lacking a means of creating a detached copy of an object graph…

[alisoftware] So what would be the equivalent of Java's copyFromRealm in Swift? Typical situation: imagine a Contacts app, with Person objects each containing an Address property. First screen is the list of contacts, tap on one opens a modal to let you modify the person and its address, with "Cancel" vs. "Validate" buttons on top

Typical workflow I'd use in this situation is to make a standalone copy of the Person to pass it to the editing modal, then use that standalone copy as the DataSource of the editing TableView to fill the various EditField cells etc… then if the user hits Cancel we drop that Person and if they hit "Validate" then we call the WebService, wait for the 200 OK, then on success save the modified, standalone Person back in the DB (using updating:true)

The idea of doing it this way is that we somehow work on a "snapshot" of the Person during the edition step and while the WebService serializes to JSON and send the request, instead of risking it to auto-update (due to a change in another thread) during the whole process. Does that make sense?

But then if we do that, then we do have an issue with person.address not being detached and risking to auto-update, despite person being detached :confused:

[bdash] Yes, I can see why it’s desirable

[jpsim]

I'd much rather have Realm object initializers do a deep copy, including relationships, than expose a new API for creating copies, but yes I think this is something we should do.

[AliSoftware]

I'd be ok with that solution too :+1:

(Or maybe even a better flexible solution, init(value: …, deepCopy: Bool = true) to let possibility of both but default to deep?)

[jpsim]

Or even init(value: …, maxDepth: Int? = nil)

[cmelchior]

The later would fit what we are doing on Android.

[olympianoiseco]

Is initWithValue: in Objective-C a deep copy? From a quick test, it appears to be.

[bdash]

No, it's a shallow copy.

[olympianoiseco]

Maybe I'm confused about what a deep copy means in this case. Here's my test code. ONCSong, ONCPad, ONChord, and ONCPitch are all RLMRealm subclasses. In this example, the NSLog statement correctly prints the values of the copied objects.

    ONCSong * _storedSong = [[ONCSong alloc] init];
    for (int i = 0 ; i < 12; i++) {
        ONCPad * pad = [[ONCPad alloc] init];
        pad.chord = [[ONCChord alloc] init];
        pad.chord.rootString = @"ad";

        for (int j = 0; j < 3; j++) {
            ONCPitch * pitch = [[ONCPitch alloc] init];
            pitch.midiValue = rand()%127;
            [pad.chord.pitches addObject:pitch];
        }

        [_storedSong.pads addObject:pad];
    }

    ONCSong * copiedSong = [[ONCSong alloc] initWithValue:_storedSong];
    NSInteger count = copiedSong.pads.count;
    for (NSInteger i = 0; i < count; i++) {
        NSLog(@"%@", @([copiedSong.pads[i].chord.pitches[0] midiValue]));
    }

[JadenGeller]

copiedSong is a new instance of the object with all the same members. Since pads is simply a pointer to a collection, that same pointer will be copied to pads property on copiedSong. This means that it will point to the same collection. If you this array were mutable, calling addObject: on one would also add it to the other.

With a deep copy, whenever a member is a pointer to another object, it will call deep copy on that object as well giving an entirely distinct set of instances. If you add an object to the array in this case, it won't affect the other array at all. There are no shared components.

tl;dr: A shallow copy only shares any objects its points to with the original. A deep copy does not. duplicates

[olympianoiseco]

Ah, thanks.

[dltlr]

+1

[AliSoftware]

Any idea on a priority/roadmal on that feature?

Here's another typical use case where we would really need that is the "Edit ViewController" use case:

• Imagine an app like some AddressBook, managing class Contact: Realm.Object objects
• I have a ContactEditViewController which allows the user to edit that Contact (edit its name, phone numbers (e.g. a 1-* relationship), etc… and that ContactEditVC has OK & Cancel buttons
• When the user taps a Contact in my ContactListViewController to edit that contact, I want to make a deep copy detachedContact = Contact(value: originalContact) of that Contact object and pass it to the ContactEditVC — so I can use that detachedContact as my (View)Model and fill the ViewController's fields with its values.
• That way, when the user change the texts in the fields, I update the detatchedContact's properties with the new value. Then only when the user hits the OK button do I do a write transaction to save the detachedContact back into the Realm with the updated values. And if the user hits Cancel, I just drop the detachedContact and ignore the changes

If I don't do a deep detachedCopy in that case and change the original object, either I won't be able to change the properties of the original object outside of a write transaction, or I'll have do to it in a write transaction and won't be able to Cancel

But if I use the current implementation of init(value:) to do that detachedCopy that will make a shallowCopy, so if the user edit the phones of the Contact (which would be a 1-* relationship here) and that copy is shallow, that will crash because the relationships would not be detached from their Realm, the copy not being a deep copy.

There really are so many use cases in an app were a deep copy is needed, that "object edition" use case is just one of them, but probably the most obvious and common one.

[SandyChapman]

It seems to me this is suggesting to do a deep copy to get around the thread access protection. Isn't the correct solution in this case to figure out a way to pass a Realm Object across threads safely? Even if needing to manually do something like:

// Thread 1
RLMObjectSubclass* obj = [RLMObjectSubclass allObjects].firstObject;

// Thread 2
RLMObjectSubclass* objForThread2 = [obj inCurrentThread];
RLMObjectSomeOtherSubclass* obj2 = [RLMObjectSomeOtherSubclass new];
obj2.property = objForThread2;
[[RLMRealm defaultRealm] transactionWithBlock:^{
    [[RLMRealm defaultRealm] addOrUpdateObject:obj2]
}];

In this example if - (instancetype)inCurrentThread is called on the same thread the object belongs to, it's a no-op and just returns self. Otherwise, it safely looks up the primary key and re-requests it on the current thread using the defaultRealm and returns it.

[AliSoftware]

@SandyChapman the use case here isn't too pass an object across threads. It is rather to do a deep copy detached from any Realm in order to create a standalone object for example.

See the detailed use case example I gave in my previous comment above: the need there is to manipulate a copy of the object rather than the original to be sure not to alter the original and/or its dependencies. (This could also be seen in a user case similar to CoreData's concept of "child managedObjectContexts")

PS: Besides, not all Realm objects have a primary key depending on your model and it's constraints so your solution wouldn't always be applicable anyway :😉

[jasper-ch-chan]

+1 Are there any updates regarding if this enhancement is being worked on actively/considered?

[jpsim]

No, this isn't highly prioritized or actively being worked on at this time. We publicly share our roadmap and priorities in GitHub issues. For example, see issues marked as S:In Progress, S:Review and S:P1 Backlog for example.

[jasper-ch-chan]

@jpsim What would have to happen in order for this enhancement to be more highly prioritized/actively worked on?

[jpsim]

All the other higher priority work would need to be done.

[shams-ahmed]

I wonder if anyone willing to do a pull request of this feature, there contributing readme file state there happy for people to submit code!

[KelvinJin]

I wonder if there's another realm alternative that would enable passing object across threads.

I know this may be too greedy but there are two features I really wish Realm could do in the future:

• The ability to change the object out of a transaction. (Potentially risky but could allow the user to decide whether to turn this on/off)
• The ability to pass the object across threads.

[anlaital]

This is a very common pattern for us as well. This is the very simple implementation we're using to create detached copies from Realm objects:

protocol DetachableObject: AnyObject {

    func detached() -> Self

}

extension Object: DetachableObject {

    func detached() -> Self {
        let detached = type(of: self).init()
        for property in objectSchema.properties {
            guard let value = value(forKey: property.name) else { continue }
            if let detachable = value as? DetachableObject {
                detached.setValue(detachable.detached(), forKey: property.name)
            } else {
                detached.setValue(value, forKey: property.name)
            }
        }
        return detached
    }

}

extension List: DetachableObject {

    func detached() -> List<Element> {
        let result = List<Element>()
        forEach {
            result.append($0.detached())
        }
        return result
    }

}

[sipersso]

@anlaital This solution works, but not if you have any circular dependencies. Do you have any strategy for this case, or do you just avoid circular dependencies?

EDIT: Circular dependencies is not a good idea. On iOS you can use backlinks. On Android I replaced them with Queries.

[zsszatmari]

+1, this would be nice.

[Alarson93]

@anlaital Detached on object works, however I get a compiler error Value of type 'Element' has no member 'detached'

Thoughts?

[austinzheng]

If you have an issue, please file a new ticket and fill out the Issue Template, rather than commenting on closed issues. Thank you!