Realm Performance With Lots of Small Objects

[nburatovich]

I've been evaluating Realm and its performance. An important use case of mine involves large numbers (100000+) of "micro objects". These are really small objects with no more than a few fields, most of which are numbers (for this test no more than 64 bytes of data total). Query performance is very good, even with high object counts, but adding, deleting, and modifying large numbers of objects from Realm can be very slow.

Test setting:

• Realm version: 0.83.
• Hardware / OS: 2.3Ghz Retina MacBook Pro / OS X 10.9.4.
• No properties are indexed or have any attributes (confirmed with attributesForProperty:).
• Using the defaultRealm.
• Setting [RLMRealm useInMemoryDefaultRealm] has no effect (not that I could run this way for production).
• Setting defaultRealm.autorefresh = NO has no effect (and it always seems to be turned on again during subsequent uses of the realm).

Add objects test: I am generating n RLMObjects, adding them to an NSArray, then timing adding that array to the realm with [realm transactionWithBlock:^{ [realm addObjectsFromArray:array]; }];

• Adding a 100000 object array: 10.891676 seconds!
• Adding a 50000 object array: 2.418253 seconds!
• Adding a 25000 object array: 0.660159 seconds.
• Adding a 10000 object array: 0.131164 seconds.

It does not seem to matter how many objects are in the realm prior to adding the objects. Adding a 10000 object array takes the same time if 0 objects are in the Realm or 1 million objects are in the Realm.

Remove objects test: I am taking n RLMObjects that exist in the Realm, adding them to an NSArray, then timing removing that array from the realm with [realm transactionWithBlock:^{ [realm deleteObjects:itemsToRemove]; }];

• Removing a 100000 object array: 10.855915 seconds!
• Removing a 50000 object array: 2.325231 seconds!
• Removing a 25000 object array: 0.597455 seconds.
• Removing a 10000 object array: 0.102262 seconds.

It's basically the same performance as the add case. Like the add case, the number of objects that exist in the Realm do not seem to impact delete performance.

Modify objects test: I am taking the first n RLMObjects from a 100000 object Realm and changing a single integer property similar to above.

• Modifying 100000 objects: 9 minutes 8.5386 seconds! (!!)
• Modifying 50000 objects: 4 minutes 36.5388 seconds! (!!)
• Modifying 25000 objects: 2 minutes 18.1554 seconds! (!!)
• Modifying 10000 objects: 55.511092 seconds! (!!)
• Modifying 5000 objects: 27.538283 seconds! (!)
• Modifying 1000 objects: 5.507278 seconds!
• Modifying 500 objects: 2.797262 seconds!
• Modifying 100 objects: 0.545877 seconds!

Taking the first n RLMObjects from a 10000 object Realm and changing a single integer property similar to above.

• Modifying 10000 objects: 5.118958 seconds!
• Modifying 5000 objects: 2.646211 seconds!
• Modifying 1000 objects: 0.525338 seconds!
• Modifying 500 objects: 0.270613 seconds!
• Modifying 100 objects: 0.056823 seconds.

The number of objects in the Realm do impact performance here. From these results, when you have 10x the objects in the Realm, the result is about 10x slower performance for the same number of objects modified in a single transaction!

Takeaways:

• When adding and removing items from a Realm, generally, using more transactions with fewer objects performs significantly faster than using fewer transitions with more objects. i.e., Add 10 batches of 10K objects instead of 1 batch of 100K objects. I have not approximated the point of diminishing returns with this. This shouldn't be. When adding an array of objects, Realm knows the total number of items it is getting and has a schema for all of them. It should be able to make decent estimates for the amount of memory it needs to allocate (and with some objects, perfect estimates) and the space it needs to free.
• Modifying is show-stopper expensive, even in a "very small" (for my case) 10K object Realm with fewer objects being modified. While I have not tried this, from the data above it seems like it would be faster to query Realm for the objects in question, delete them, recreate new instances with the correct properties, and add them, combining it with the multi-batch approach mentioned above. This should not be the case. That's more work (and more needless notifications generated).
• Deleting objects taking as long as adding objects - I don't know what to think. It is not vacuuming the Realm file.

Other notes:

• Memory use remained acceptable throughout the testing.

I generally like Realm's API and how to thinks about things even with what it has missing. I'd like to use it, but it doesn't seem particularly suited for this use case. Thoughts about what can be done here or what the timeline is for improvements?

[bmunkholm]

Hi, Thanks a lot for taking the time to make and provide the observations! Realm has at this point primarily been optimized for query performance. So we expect to optimize a lot of the operations you refer to going forward. The good thing is that the core database technology is very fast, and the challenge is that adding the nice object abstraction and input validation eats up a lot of performance. And the last part has not really been optimized yet.

I'm not sure exactly how you do the modification test, could you share more detail on that? Even better if you could send me the benchmark code, so we can see if there is anything easy to do better.

It could also be helpful to understand your usecase a little better to see if a solution could be to provide either a more lower level API or maybe some batch update API.

You are welcome to send any details or code in confidentiality to help@realm.io.

Thanks!

[tgoyne]

I suspect that the bottleneck in some of your tests (insertion and deletion, but probably not mutation) is that deallocation of attached RLMObjects currently takes time proportional to the number of objects alive for the backing table view, and so avoiding keeping large numbers of objects alive at a time may make things significantly faster. E.g. for insertions, constructing and inserting each object immediately rather than building up an array (with an autoreleasepool around the body of the loop if needed) or possibly even just removing each item from the array after inserting it (so that there's only a single attached object at a time) may help.

If that is in fact the problem, I have a partially working solution that I hadn't been prioritizing finishing up due to not being sure if it was an actual problem for users.

[nburatovich]

https://github.com/nburatovich/Realm-Micro-Benchmark shows what I'm testing.

The modification benchmark is nothing more than something like:

RLMArray *locations = [MicroObject allObjectsInRealm:realm];
NSUInteger count = locations.count;
[realm transactionWithBlock:^{
    for (NSUInteger index = 0; index < count; index++) {
        MicroObject *location = locations[index];
        location.identifier = location.identifier + 1;
    }
}];

Except usually modifying a much smaller subset of objects in a Realm with many objects (not all of them).

My use case is that these “micro objects” serve as proxies for much larger, more detailed objects. Their limited info can be presented in a search summary, in other views, and as aggregate results, while larger, more detailed objects are batch-fetched in the background from a server (when drilling down into deeper detail).

Initially, there will be between 125-150K of these objects. Over time, it won’t be unusual to need to modify or add anywhere between 100 - 2000 objects during a typical app launch (more if atypical). If updates are available, that’s a small packet to send given the small objects we are dealing with. Essentially, it’s a list that will be kept in sync with delta updates from our server.

I may be better off not using Realm in this case given my results, but I figured I'd try since it would save me some effort. The overall data set in this case is very small - about 6 MB of data total and ~125-150K rows (which is not a lot for a database). From other tests, using NSKeyedArchiving and reading everything into memory is relatively fast, although memory use can spike quite a bit, I need to deal with threading issues, and using NSPredicate on an NSArray (or even filtering through enumeration and if-statements) is slower than Realm.

[tgoyne]

With #797 merged, using fast enumeration (i.e. for (MicroObject *location in locations) { ... }) makes the mutation benchmark with 10k items go from 6.101217 seconds to 0.012706 seconds for me. The key difference is that since modifying the set of items being iterated over during iteration is illegal it's able to skip that check, while when iterating over the indexes it has to check if the mutation removed the item from the query.

There's a pending change to the core which drops the insertion benchmark with 10k items from 0.150226 for me to 0.042819 second for me.

[tgoyne]

The aforementioned core improvement which speeds up insertion is in now in master.