With Live Paged data delays are experienced

[beczesz]

Hi Gábor,

First of all thank you for this excellent lib, it game me more insight to Realm and I started to love it too. I'm developing an Android email client and now I try to optimize the message list for situations when we have x00k messages in the database. My current problem is that any changes done in the background takes like 6-7 seconds to appear on the UI. To query the message list itself takes like 6-7 seconds.

I was hoping before I implement a manual solution to this that maybe Monarchy with its Paged LiveData would give a solution but now playing around with your example I generated 800k random dogs and sorted based on their names so it takes like 11 seconds to query it.

If I change one of the names it still takes like 8 seconds to update the UI. Am I missing the point of LiveData?

What would be your approach?

Thank you, Szabolcs

[Zhuinden]

Heya,

The issue is on Realm side not supporting SKIP/TAKE, only LIMIT.

This means that in order to obtain a page of results, the full data set needs to be evaluated before you can slice out a window... 😞

So the paging integration currently allows you to not load all 10k+ results in the list that you are displaying, but it doesn't stop Realm from trying to figure out how to access 10k results if you were to index them.

There is a good chance that updating Monarchy to support the emission of a LiveData<List<T>> as a Frozen RealmResults (Realm-Java 6.1.0) would provide better results than the current paging query, but it would still take Realm the time to start retrieving objects.

Honestly, Realm scales well up to ~100k objects, but beyond that it becomes noticeably slow when you do complex queries on non-indexed fields.

You would need OFFSET described in https://github.com/realm/realm-java/pull/6126 but I don't think that ever happened.

So yeah, the real question is how it would behave using the new realmResults.freeze() API, instead of using copied.

What happens if you try findAllManaged()?

[beczesz]

Thanks for the quick response. I'm a bit mixed up between the versions now. Isn't Frozen RealmResults a feature which is introduced in v7.x? I updated to 6.1.0 but I see no freeze function.

Anyways I think I understand why you are asking it. I already did a workaround which essentially freezes the state of the object, passes it to the UI thread and temporarily overrides the UI state until the change event arrives.

This is one way to optimize the UX. the other way is to optimize the query itself. Although this question is not strictly related with the opened issues I would ask it here, but if you would like to I can open an another one. In fact there is one opened already here on Stackoverflow and [here on the Realm forums] (https://forum.realm.io/t/slow-updates-in-case-of-400k-result/3559)

My query is slow because of sorting the results. Without sorting it would be ~200 ms with sorting is like 5 seconds. I tried to index the field based on which I sort but I saw no performance upgrade. Am I doing something wrong or this is expected?

The other optimization possibility as I see is to maintain the order of the objects already sorted, so I would query it without sorting it.

The third possibility would be to make sure that the query results would not contain more than several thousands of items at most.

By the way finsAllManaged is faster but it does not addresses the real issue, namely how to update the UI instantaneously (or within ~100-200ms) if I change an object in the background.

[Zhuinden]

I somewhat forgot about this issue (sorry!) although there's also the side of things that without SKIP/TAKE support on Realm side, the Realm query on change will evaluate the full result set, and that will take a lot of time.

If even findAllManaged takes long, then the slowness comes from Realm itself, and there isn't really any means of optimization that I could add as a wrapper.

One possible thing to note is that query speed is drastically reduced if the Realm schema contains bidirectional links. That was the recommendation up to 3.2.0, after which they were a performance problem and had to be replaced with uni-directional link + @LinkingObjects (inverse relation).

[Zhuinden]

With Realm 7.0.0's support for frozen results, it can make sense to use the findAllFrozenWithChanges method over the findAllPagedWithChanges method, as Realm doesn't really support skip+take anyway.