Object not up to date in change listener

[laurentmorvillier]

How frequently does the bug occur?

Always

Description

I have a singleton TimeManager that permanently listens to changes in my Session to perform backoffice stuff.

In the code below, I'm adding a TimeInterval object to the timeIntervals RealmList of my Session object after deleting the existing ones, each time a Session changes.

On a first change triggered by the UI, everything is fine and my Session has one TimeIntervalobject.

On a second change triggered by the UI, what's going wrong is that nothing is deleted when I do session.timeIntervals.deleteAllFromRealm() as session.timeIntervals is empty, as the Timber log shows.

So it looks like the timeIntervals RealmList is not up to date, but calling realm.refresh() does not solve the issue. If I restart my app between the two changes, everything is fine.

I've tried to add an onSuccess listener just to see if it was called and it does. There is pretty much no lag as I'm doing this on an empty Realm.

Here is the code:

object TimeManager {

    var sessions: RealmResults<Session>? = null

    private val sessionIdsToProcess = mutableSetOf<String>()

    fun configure() {} // launch init

    fun sessionChanged(session: Session) {
        this.sessionIdsToProcess.add(session.id)
    }

    init {

        val realm = Realm.getDefaultInstance()

        sessions = realm.where(Session::class.java).findAllAsync()
        sessions?.addChangeListener { _, _ ->

            if (sessionIdsToProcess.isNotEmpty()) {

                realm.executeTransactionAsync({ asyncRealm ->

                    val sessions = sessionIdsToProcess.mapNotNull { asyncRealm.findById<Session>(it) }
                    sessionIdsToProcess.clear()

                    for (session in sessions) {
                        Timber.d("Session id = ${session.id}")
                        Timber.d("Session time intervals count = ${session.timeIntervals.size}")
                        session.timeIntervals.deleteAllFromRealm()
                        val fti = TimeInterval()
                        session.timeIntervals.add(fti)
                        asyncRealm.insertOrUpdate(session)
                    }

                }, {
                    Timber.d("executeTransactionAsync onSuccess listener...")
                    val timeIntervals = realm.where(TimeInterval::class.java).findAll()
                    Timber.d("Total timeIntervals count = ${timeIntervals.size}")

                    timeIntervals.forEach {
                        Timber.d(">>> Time interval session count = ${it.sessions?.size}, session id = ${it.sessions?.firstOrNull()?.id}")
                    }

                }, {})
            }
        }
    }
}

class MainActivity : AppCompatActivity() {

    [...]

    // action triggered by a button
    private fun updateSession() {

        Timber.d("Update session")

        session.size = Random.nextInt()
        TimeManager.sessionChanged(session)

        realm.executeTransactionAsync {
            it.insertOrUpdate(this.session)
        }

    }

}

open class Session : RealmObject() {

    @PrimaryKey
    var id = UUID.randomUUID().toString()

    var size: Int? = null

    var timeIntervals: RealmList<TimeInterval> = RealmList()

}

open class TimeInterval : RealmObject() {

    @PrimaryKey
    var id = UUID.randomUUID().toString()

    @LinkingObjects("timeIntervals")
    val sessions: RealmResults<Session>? = null

}

Stacktrace & log output

Here are the Timber logs:

First change:

    TimeManager: Session id = d7da4b37-c0e0-465f-8a45-69cc7753ea1e
    TimeManager: Session time intervals count = 0
    TimeManager: executeTransactionAsync onSuccess listener...
    TimeManager: Total timeIntervals count = 1
    TimeManager: Time interval session count = 1, session id = d7da4b37-c0e0-465f-8a45-69cc7753ea1e

Second change:

    TimeManager: Session id = d7da4b37-c0e0-465f-8a45-69cc7753ea1e TimeManager: Session time intervals count = 0
    TimeManager: Total timeIntervals count = 2
    TimeManager: Time interval session count = 0, session id = null
    TimeManager: Time interval session count = 1, session id = d7da4b37-c0e0-465f-8a45-69cc7753ea1e

Now if I stop my app after the first change and launch it again, here are the logs I get for the second change (the id changes because I started from scratch):

    TimeManager: Session id = 534e9294-6462-4efa-aaf9-834d085aa2ba TimeManager: Session time intervals count = 1
    TimeManager: executeTransactionAsync onSuccess listener...
    TimeManager: Total timeIntervals count = 1
    TimeManager: >>> Time interval session count = 1, session id = 534e9294-6462-4efa-aaf9-834d085aa2ba

Can you reproduce the bug?

Always

Reproduction Steps

Here is a github project to reproduce the issue: git@github.com:laurentmorvillier/realmtest.git

When installing the app, tap on the floating button to make a first change to the Session, then tap a second time to make a second change.

Version

10.13.0

What Atlas App Services are you using?

Local Database only

Are you using encryption?

No

Platform OS and version(s)

32/33

Build environment

Android Studio version: Android Studio Chipmunk | 2021.2.1 Patch 1 Build #AI-212.5712.43.2112.8609683, built on May 18, 2022 Runtime version: 11.0.12+0-b1504.28-7817840 aarch64 VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o. macOS 13.0.1 GC: G1 Young Generation, G1 Old Generation Memory: 3072M Cores: 8 Registry: external.system.auto.import.disabled=true, ide.instant.shutdown=false Non-Bundled Plugins: org.intellij.plugins.markdown (212.5457.16), org.jetbrains.kotlin (212-1.7.0-release-281-AS5457.46)

Android Build Tools version: ... Gradle version: 7.3.3

[edualonso]

Hi @laurentmorvillier. We can reproduce your problem. We will get back to you once we have more news.

[edualonso]

@laurentmorvillier there is a problem with this code in MainActivity.updateSession():

        realm.executeTransactionAsync {
            it.insertOrUpdate(this.session)
        }

You are reusing MainActivity.session when you call updateSession(). This session is instantiated only once with an empty list for the timeIntervals field. When you call insertOrUpdate with that object, Realm will update the object that matches the primary key you defined with its default value, but the timeIntervals field gets overwritten by the default value present in said instance.

You can avoid this problem by querying the current session or instantiating a new one when you press the button for the first time:

    private fun updateSession() {
        Timber.d("------ Update session")
        realm.executeTransactionAsync {
            val queriedSession = it.where(Session::class.java)
                .findFirst() ?: Session()
            queriedSession.size = Random.nextInt()

            TimeManager.sessionChanged(queriedSession)
            it.insertOrUpdate(queriedSession)
        }
    }

[laurentmorvillier]

@edualonso Thanks a lot for helping out.

This leads to a critical architectural issue for me: My Session properties are displayed in the UI where the user can make some changes (like session.size = Random.nextInt()). I create this local copy so that when the user changes something, the UI is instantly up to date.

When I make the change inside the asynchronous transaction, my local copy is not up to date anymore, making the UI unaware of the change. If I wait for the onSuccess to copy the updated Session, I'll have a lag in my UI because the writing can take some time.

What is the best practice here?

[edualonso]

I create this local copy so that when the user changes something, the UI is instantly up to date.

But remember that you also have to update your local copy with the modifications you make in your asynchronous transactions, otherwise you will have inconsistent information in your local copy and what you fetch from your realm, which in the case is the very problem you are experiencing.

Generally, I would advise against keeping a local copy since you already have your data in your realm, although you can make it work as long as you keep it up to date, including timeIntervals, with updated values - that is, you would have to query your realm to get the updated intervals before updating the object in your realm with insertOrUpdate if you use those intervals in your UI. If you don't do that, you will be overwriting them with the default value from your model class every time you write it to your realm.

Your approach of using a change listener in TimeManager is just fine. I would investigate how to bind those results you receive in the change listener callback to your UI, for example using viewmodels (https://developer.android.com/topic/libraries/architecture/viewmodel).

I will close the issue since this is a matter of implementation details and not a bug.

[laurentmorvillier]

I'd be happy avoiding making two times the same change to the local copy and the managed object, but I'm unsure how to do otherwise as you would advise. When a user makes a changes with the UI, it needs to be reflected instantly (I'm not talking about changes made by the TimeManager listener, just direct object update). My app can deal with heavy volume and a write can take seconds. Is there a way to achieve this without a local object?

[edualonso]

TLDR; You need to find the sweet spot between "many manageable transactions vs few large transactions" and that depends on your business logic and how heavy your dataset is. There are multiple ways to bind your UI to your data models. As suggested in the previous reply, you could use viewmodels combined with an architectural pattern like MVVM.

If your writes are manageable enough in size you should, in principle, not experience latency issues - "manageable" is ambiguous but that is because there is not a golden number that can tell you from which point things become slower.

If you, on the other hand, are working with vast amounts of data (tens or hundreds of thousands or even millions of objects) in your transactions, or alternatively run lots of small transactions instead of making large, batch updates in one single transaction, you should expect slowdowns. Realm is primarily designed to work with small-to-medium datasets that can run in a wide range of mobile devices, some with lots of memory, some with much less.

If your update logic takes up seconds you should definitely re-evaluate your approach to how you are writing your data. You could try to write small, batch, asynchronous updates and reflect those updates on your UI after you get each change listener callback with the updated results - remember you also get a changeset object that tells you which elements have changed so that you can optimise your own UI update logic and only change those positions that need updates instead of the whole dataset. In that way you would avoid waiting for a long time until everything is written and would get progressive updates instead and your performance wouldn't stall.

[laurentmorvillier]

In my app I usually make very small transactions, like just writing in one field of an object. When my app has about 10,000 records in a few entities I have a writing duration of 1-2 seconds, which seems unexpected from what you are saying, but I'm working with a low end device, so could it be the reason? If the time of to reach the onSuccess is negligible, I could consider copying my managed object when the onSuccess is called. How can I make sure I reach peak performance?