search

realm / realm-swift

Realm is a mobile database: a replacement for Core Data & SQLite
https://realm.io
Apache License 2.0
16.3k stars 2.14k forks source link

Realm triggers modification change notifications where there should be none. #3494

Closed kunalsood closed 8 years ago

kunalsood commented 8 years ago

Goals

As an example, an RSS Reader App, which has a Feed object, and a FeedItem object. Feed has a to-many relationship items to FeedItem. FeedItem has a to-one relationship feed to Feed.

Every time I do a feed refresh, new feed items are downloaded, and imported into the database. Both relationships are created explicitly.

My goal is to display FeedItem objects to the user, using RLMResults to populate my UI, and make animated changes to the UI when new items are downloaded using RLMResults' change notifications API.

Expected Results

RLMResults change notifications will only include change-sets for items that have been inserted, deleted, or modified.

Actual Results

RLMResults change notifications includes modification change-sets for every FeedItem object already in the database before the write transaction that added new objects was committed.

Steps to Reproduce

Create new "Single View Application" project in Xcode, follow the steps to add Realm framework to the project, copy & paste the code sample below into the root view controller, run the app & watch the logs. First it will produce logs like this..

2016-04-28 13:55:09.682 RLMIssueSample[10785:4734055] 0 results at first load
2016-04-28 13:55:09.688 RLMIssueSample[10785:4734055] Staring Import
2016-04-28 13:55:09.738 RLMIssueSample[10785:4734097] Committed Write Transaction, Saved 100 total items in Realm
2016-04-28 13:55:09.739 RLMIssueSample[10785:4734055] 0 deletions, 100 insertions, 0 modifications, 100 total objects

Which is fine as there was nothing in the DB yet, so, no modification change-sets. But wait for the next set of change-set notifications & consequential logs. Which will show something like...

2016-04-28 13:55:09.804 RLMIssueSample[10785:4734097] Committed Write Transaction, Saved 200 total items in Realm
2016-04-28 13:55:09.811 RLMIssueSample[10785:4734055] 0 deletions, 100 insertions, 100 modifications, 200 total objects
2016-04-28 13:55:09.834 RLMIssueSample[10785:4734097] Committed Write Transaction, Saved 300 total items in Realm
2016-04-28 13:55:09.867 RLMIssueSample[10785:4734055] 0 deletions, 100 insertions, 200 modifications, 300 total objects
2016-04-28 13:55:09.899 RLMIssueSample[10785:4734097] Committed Write Transaction, Saved 400 total items in Realm
2016-04-28 13:55:09.931 RLMIssueSample[10785:4734055] 0 deletions, 100 insertions, 300 modifications, 400 total objects
2016-04-28 13:55:10.107 RLMIssueSample[10785:4734097] Committed Write Transaction, Saved 500 total items in Realm
2016-04-28 13:55:10.108 RLMIssueSample[10785:4734055] 0 deletions, 100 insertions, 400 modifications, 500 total objects
2016-04-28 13:55:10.152 RLMIssueSample[10785:4734097] Committed Write Transaction, Saved 600 total items in Realm
2016-04-28 13:55:10.153 RLMIssueSample[10785:4734055] 0 deletions, 100 insertions, 500 modifications, 600 total objects
2016-04-28 13:55:10.190 RLMIssueSample[10785:4734097] Committed Write Transaction, Saved 700 total items in Realm
2016-04-28 13:55:10.191 RLMIssueSample[10785:4734055] 0 deletions, 100 insertions, 600 modifications, 700 total objects
2016-04-28 13:55:10.209 RLMIssueSample[10785:4734097] Committed Write Transaction, Saved 800 total items in Realm
2016-04-28 13:55:10.210 RLMIssueSample[10785:4734055] 0 deletions, 100 insertions, 700 modifications, 800 total objects
2016-04-28 13:55:10.296 RLMIssueSample[10785:4734097] Committed Write Transaction, Saved 900 total items in Realm
2016-04-28 13:55:10.298 RLMIssueSample[10785:4734055] 0 deletions, 100 insertions, 800 modifications, 900 total objects
2016-04-28 13:55:10.335 RLMIssueSample[10785:4734097] Committed Write Transaction, Saved 1000 total items in Realm
2016-04-28 13:55:10.337 RLMIssueSample[10785:4734055] 0 deletions, 100 insertions, 900 modifications, 1000 total objects

Notice the number of modifications being reported here.

Code Sample


#import <Realm/Realm.h>

@class Feed;

@interface FeedItem : RLMObject
@property NSString *uuid;
@property Feed *feed;
@end
RLM_ARRAY_TYPE(FeedItem)

@interface Feed : RLMObject
@property NSString *uuid;
@property RLMArray<FeedItem *><FeedItem> *items;
@end
RLM_ARRAY_TYPE(Feed)

@implementation FeedItem

+ (NSArray *)requiredProperties
{
    return @[@"uuid"];
}

+ (NSDictionary *)defaultPropertyValues
{
    return @{@"uuid": [[NSUUID UUID] UUIDString]};
}

+ (NSString *)primaryKey
{
    return @"uuid";
}

@end

@implementation Feed

+ (NSArray *)requiredProperties
{
    return @[@"uuid"];
}

+ (NSDictionary *)defaultPropertyValues
{
    return @{@"uuid": [[NSUUID UUID] UUIDString]};
}

+ (NSString *)primaryKey
{
    return @"uuid";
}

@end

@interface ViewController ()

@property (nonatomic, strong) RLMResults *results;
@property (nonatomic, strong) RLMNotificationToken *notification;

@end

@implementation ViewController

- (void)viewDidLoad
{
    [super viewDidLoad];
    __weak typeof(self) weakSelf = self;
    self.notification = [[FeedItem allObjects] addNotificationBlock:^(RLMResults * _Nullable results, RLMCollectionChange * _Nullable change, NSError * _Nullable error) {
        if (change)
        {
            NSLog(@"%@ deletions, %@ insertions, %@ modifications, %@ total objects",
                  @(change.deletions.count),
                  @(change.insertions.count),
                  @(change.modifications.count),
                  @(results.count));
        }
        else
        {
            weakSelf.results = results;
            NSLog(@"%@ results at first load", @(weakSelf.results.count));
            [weakSelf startImport];
        }
    }];
}

- (void)didReceiveMemoryWarning
{
    [super didReceiveMemoryWarning];
}

- (void)startImport
{
    NSLog(@"Staring Import");
    dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
        RLMRealm *realm = [RLMRealm defaultRealm];
        NSUInteger batch = 0;
        NSUInteger total = 50000;
        [realm beginWriteTransaction];
        Feed *newFeed = [[Feed alloc] init];
        [realm addObject:newFeed];
        for (NSUInteger i = 0; i < total; i++)
        {
            FeedItem *newItem = [[FeedItem alloc] init];
            newItem.feed = newFeed;
            [newFeed.items addObject:newItem];

            batch++;
            if (batch == 100)
            {
                batch = 0;
                [realm commitWriteTransaction];
                NSLog(@"Committed Write Transaction, Saved %@ total items in Realm", @(i+1));
                if (i < (total-1))
                {
                    [realm beginWriteTransaction];
                }
            }
            else if (i == (total-1))
            {
                [realm commitWriteTransaction];
                NSLog(@"Committed Write Transaction, Saved %@ total items in Realm", @(i+1));
            }
        }
    });
}

@end

Version of Realm and Tooling

Realm version: 0.99.1

Xcode version: 7.3

iOS/OSX version: 9.3.1

Dependency manager + version: N/A (Downloaded binaries from http://realm.io & dragged the static framework into my Xcode project)

tgoyne commented 8 years ago

Modifications to linked objects are counted as also being modifications to the linking object, so in this sample every object is being modified as feed.items has changed. Using linkingObjectsOfClass:forProperty: or the upcoming linking object property functionality rather than manually tracking the bidirectional relationship would avoid that behavior, although that could be considered a bug.

kunalsood commented 8 years ago

Thanks for your response!

although that could be considered a bug

So, I presume this (modifications to linked objects also being counted as modifications to the linking object) is a design decision? I'm not going to try and convince you to change it if it is, but may I ask if there is any particular reason why notifications were designed this way? I ask because this is not how things work in Core Data/NSFetchedResultsController (at least they didn't the last time I used it). (Though the fact that I didn't manually manage bi-directional relationships in Core Data, and that it returns NSSets for to-many relationships could have something to do with that.)

Backlinking via linkingObjectsOfClass:forProperty: is not for me as I need the ability to query this relationship. Though the upcoming linking object property looks promising, you saying that their current behaviour (w.r.t. notifications) could be considered a bug, does not give me any confidence to use them at all.

Further I should add, this sample has a rather simple model object graph. In a real world application, with a more complex object graph (where objects have more properties & more manually managed bi-directional relationships), I have noticed Realm spending way too much time & using too much CPU resources in RLMRealm notification listener thread, (I'm guessing) computing change sets due to changes in linked objects (though my guess could be wrong).

jpsim commented 8 years ago

I ask if there is any particular reason why notifications were designed this way? I ask because this is not how things work in Core Data/NSFetchedResultsController

What's a little funny is that this limitation in Core Data was part of the motivation for designing it this way 😆. Say you had a UITableView of email messages, and you wanted to show the sender's name in the cells:

class Person: Object {
  dynamic var name = ""
}
class Email: Object {
  dynamic var sender: Person?
}

you'd want to know when that cell had a relationship object that had changed. The way to do this in Core Data would be incredibly hacky and underperformant...

We considered adding the ability to limit change notifications by observing specific keypaths, but opted against it in the end due to the additional complexity of that API, and that performance while tracking all keypaths was good enough. This is something we're open to doing in the future...

The performance issue you bring up is concerning, and we'd appreciate sample projects that could demonstrate real world performance issues so we can investigate and hopefully optimize.

tgoyne commented 8 years ago

A cycle in the graph where one of the nodes is a 50k element list would definitely hit performance issues in modification checking if there's a modification to an object of the same type that's not in the SCC being checked. Need to actually track the nodes that have been visited rather than just capping the search depth I guess.

kunalsood commented 8 years ago

this limitation in Core Data was part of the motivation for designing it this way

Funny, and ironic. 😄 What you call a limitation, I feel is an excellent feature (or lack there of). There were way too many times that I used this feature, and replaced properties with relationships in my NSManagedObjects, just to get NSFetchedResultsController to shut up.

In my experience, when dealing with the kind of massive UITableView or UICollectionView which display tens of thousands of items, and have even slightly moderate complexity (eg. lets say variable heights on cells), the less frequently you update them the better (on older devices updating even a single row in such a tableView with just 2-3 thousand existing rows can be expensive). Which means not updating them when a random property (that you're not even showing in the cells) updates.

You're absolutely right in saying that being notified of a change to a related object in Core Data would be incredibly hacky and underperformant. But, the beauty of Realm is that it is not Core Data, reading persisted properties & traversing relationships is so fast that I pretty much never have to worry about performance. While using Realm, I do in-memory filtering on tens of thousands of objects. Something that I wouldn't even dream of doing in Core Data. My point is that, if Realm did not provide notifications for changes in related objects, and I needed them, my workaround would be hacky, but not underperformant.

It might take me a little while, but I will be happy to provide you with a sample project that demonstrates the performance issue that I brought up.

Finally, I can pretty much confirm that it was the manually managed bi-directional relationships that was the cause of excessive CPU usage in the notification listener thread (and the un-needed modification notification as already confirmed by @tgoyne). I changed my models to use the newly merged linking object property, and both issues have vanished.

At this point, my only request for you guys would be: If you ever decide to address this bug of modification notifications not being posted for 'linking object properties', please first consider adding, at least as an option, the ability to disable notifications for changes to related objects.

PS: Again, I will try and provide a sample project which demonstrates the performance issue that I brought up, as soon as I can.

kunalsood commented 8 years ago

I finally have some code for you. Here is what to do: Create new "Single View Application" project in Xcode, follow the steps to add Realm framework to the project, copy & paste the code sample below into the root view controller, run the app on an actual device, wait for it to finish the import while watching the logs.

#import <Realm/Realm.h>

RLM_ARRAY_TYPE(Feed)
RLM_ARRAY_TYPE(FeedItem)
RLM_ARRAY_TYPE(FeedFolder)

@interface FeedFolder : RLMObject
@property NSString *uuid;
@property NSString *stringProp1;
@property NSString *stringProp2;
@property NSString *stringProp3;
@property NSString *stringProp4;
@property BOOL boolProp1;
@property BOOL boolProp2;
@property RLMArray<Feed> *feeds;
@end

@interface Feed : RLMObject
@property NSString *uuid;
@property NSString *stringProp1;
@property NSString *stringProp2;
@property NSString *stringProp3;
@property NSString *stringProp4;
@property BOOL boolProp1;
@property BOOL boolProp2;
@property RLMArray<FeedItem> *items;
@property RLMArray<FeedFolder> *folders;
@end

@class Feed;

@interface FeedItem : RLMObject
@property NSString *uuid;
@property NSString *stringProp1;
@property NSString *stringProp2;
@property NSString *stringProp3;
@property NSString *stringProp4;
@property BOOL boolProp1;
@property BOOL boolProp2;
@property Feed *feed;
@end

@implementation FeedFolder

+ (NSArray *)requiredProperties
{
    return @[@"uuid", @"stringProp1", @"stringProp2", @"stringProp3", @"stringProp4", @"boolProp1", @"boolProp2"];
}

+ (NSDictionary *)defaultPropertyValues
{
    return @{@"uuid": [[NSUUID UUID] UUIDString],
             @"stringProp1": [[NSUUID UUID] UUIDString],
             @"stringProp2": [[NSUUID UUID] UUIDString],
             @"stringProp3": [[NSUUID UUID] UUIDString],
             @"stringProp4": [[NSUUID UUID] UUIDString],
             @"boolProp1": @YES,
             @"boolProp2": @NO};
}

+ (NSString *)primaryKey
{
    return @"uuid";
}

@end

@implementation Feed

+ (NSArray *)requiredProperties
{
    return @[@"uuid", @"stringProp1", @"stringProp2", @"stringProp3", @"stringProp4", @"boolProp1", @"boolProp2"];
}

+ (NSDictionary *)defaultPropertyValues
{
    return @{@"uuid": [[NSUUID UUID] UUIDString],
             @"stringProp1": [[NSUUID UUID] UUIDString],
             @"stringProp2": [[NSUUID UUID] UUIDString],
             @"stringProp3": [[NSUUID UUID] UUIDString],
             @"stringProp4": [[NSUUID UUID] UUIDString],
             @"boolProp1": @YES,
             @"boolProp2": @NO};
}

+ (NSString *)primaryKey
{
    return @"uuid";
}

@end

@implementation FeedItem

+ (NSArray *)requiredProperties
{
    return @[@"uuid", @"stringProp1", @"stringProp2", @"stringProp3", @"stringProp4", @"boolProp1", @"boolProp2"];
}

+ (NSDictionary *)defaultPropertyValues
{
    return @{@"uuid": [[NSUUID UUID] UUIDString],
             @"stringProp1": [[NSUUID UUID] UUIDString],
             @"stringProp2": [[NSUUID UUID] UUIDString],
             @"stringProp3": [[NSUUID UUID] UUIDString],
             @"stringProp4": [[NSUUID UUID] UUIDString],
             @"boolProp1": @YES,
             @"boolProp2": @NO};
}

+ (NSString *)primaryKey
{
    return @"uuid";
}

@end

@interface RLMResults (Random)

- (id)randomObject;

@end

@implementation RLMResults (Random)

-(id)randomObject
{
    NSUInteger resultsCount = [self count];
    if (resultsCount)
        return [self objectAtIndex:arc4random_uniform((uint32_t) resultsCount)];
    else
        return nil;
}

@end

@interface ViewController ()

@property (nonatomic, strong) RLMResults *results;
@property (nonatomic, strong) RLMNotificationToken *notification;

@end

@implementation ViewController

- (void)viewDidLoad
{
    [super viewDidLoad];
    __weak typeof(self) weakSelf = self;
    self.notification = [[FeedItem allObjects] addNotificationBlock:^(RLMResults * _Nullable results, RLMCollectionChange * _Nullable change, NSError * _Nullable error) {
        if (change)
        {
            NSLog(@"%@ deletions, %@ insertions, %@ modifications, %@ total items in results",
                  @(change.deletions.count),
                  @(change.insertions.count),
                  @(change.modifications.count),
                  @(results.count));
        }
        else
        {
            weakSelf.results = results;
            NSLog(@"%@ items in results at first load", @(weakSelf.results.count));
            [weakSelf startImport];
        }
    }];
}

- (void)didReceiveMemoryWarning
{
    [super didReceiveMemoryWarning];
}

- (void)startImport
{
    NSLog(@"Staring Import");
    dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
        RLMRealm *realm = [RLMRealm defaultRealm];

        //Create 10 Folders
        [realm beginWriteTransaction];
        for (NSUInteger i = 0; i < 10; i++)
        {
            FeedFolder *newFolder = [[FeedFolder alloc] init];
            [realm addObject:newFolder];
        }
        [realm commitWriteTransaction];
        RLMResults *allFolders = [FeedFolder allObjects];

        //Create 20 Feeds (Manually creating bi-directional relationsips with 2 random folders each)
        [realm beginWriteTransaction];
        for (NSUInteger i = 0; i < 20; i++)
        {
            Feed *newFeed = [[Feed alloc] init];

            FeedFolder *randomFolder1 = [allFolders randomObject];
            [randomFolder1.feeds addObject:newFeed];
            [newFeed.folders addObject:randomFolder1];

            FeedFolder *randomFolder2 = [allFolders randomObject];
            [randomFolder2.feeds addObject:newFeed];
            [newFeed.folders addObject:randomFolder2];
        }
        [realm commitWriteTransaction];
        RLMResults *allFeeds = [Feed allObjects];

        //Create 5000 FeedItems (Manually creating bi-directional relationsips with 1 random feed each)
        NSUInteger batch = 0;
        NSUInteger total = 5000;
        [realm beginWriteTransaction];
        for (NSUInteger i = 0; i < total; i++)
        {
            FeedItem *newItem = [[FeedItem alloc] init];
            Feed *randomFeed = [allFeeds randomObject];
            newItem.feed = randomFeed;
            [randomFeed.items addObject:newItem];

            batch++;
            if (batch == 100)
            {
                [realm commitWriteTransaction];
                NSLog(@"Saved %@ new items in Realm", @(batch));
                batch = 0;
                if (i < (total-1))
                {
                    [realm beginWriteTransaction];
                }
                if (i == (total-1))
                {
                    NSLog(@"Finished saving %@ total new items in Realm", @(total));
                }
            }
            else if (i == (total-1))
            {
                [realm commitWriteTransaction];
                NSLog(@"Saved %@ new items in Realm", @(batch));
                batch = 0;
                NSLog(@"Finished saving %@ total new items in Realm", @(total));
            }
        }
    });
}

@end

The first time you run the app, the logs will look something like this

2016-05-01 12:16:05.007 RLMIssueSample[12391:5535908] 0 items in results at first load
2016-05-01 12:16:05.007 RLMIssueSample[12391:5535908] Staring Import
2016-05-01 12:16:05.133 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:05.134 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 0 modifications, 100 total items in results
2016-05-01 12:16:05.260 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:05.269 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 100 modifications, 200 total items in results
2016-05-01 12:16:05.397 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:05.455 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 200 modifications, 300 total items in results
2016-05-01 12:16:05.601 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:05.602 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 300 modifications, 400 total items in results
2016-05-01 12:16:05.799 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:05.800 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 400 modifications, 500 total items in results
2016-05-01 12:16:05.943 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:05.949 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 500 modifications, 600 total items in results
2016-05-01 12:16:06.083 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:06.095 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 600 modifications, 700 total items in results
2016-05-01 12:16:06.157 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:06.213 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 700 modifications, 800 total items in results
2016-05-01 12:16:06.308 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:06.339 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 800 modifications, 900 total items in results
2016-05-01 12:16:06.409 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:06.469 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 900 modifications, 1000 total items in results
2016-05-01 12:16:06.474 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:06.488 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 1000 modifications, 1100 total items in results
2016-05-01 12:16:06.501 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:06.503 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 1100 modifications, 1200 total items in results
2016-05-01 12:16:06.563 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:06.600 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:06.676 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:06.682 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 1200 modifications, 1300 total items in results
2016-05-01 12:16:06.737 RLMIssueSample[12391:5535908] 0 deletions, 200 insertions, 1300 modifications, 1500 total items in results
2016-05-01 12:16:06.759 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:06.762 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 1500 modifications, 1600 total items in results
2016-05-01 12:16:06.800 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:06.829 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:06.866 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:06.951 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:07.012 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:07.060 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:07.101 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:07.162 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 1600 modifications, 1700 total items in results
2016-05-01 12:16:07.177 RLMIssueSample[12391:5535908] 0 deletions, 600 insertions, 1700 modifications, 2300 total items in results
2016-05-01 12:16:07.191 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:07.194 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 2300 modifications, 2400 total items in results
2016-05-01 12:16:07.268 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:07.272 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 2400 modifications, 2500 total items in results
2016-05-01 12:16:07.314 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:07.318 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 2500 modifications, 2600 total items in results
2016-05-01 12:16:07.363 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:07.374 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 2600 modifications, 2700 total items in results
2016-05-01 12:16:07.397 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:07.470 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 2700 modifications, 2800 total items in results
2016-05-01 12:16:07.487 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:07.501 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 2800 modifications, 2900 total items in results
2016-05-01 12:16:07.537 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:07.563 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 2900 modifications, 3000 total items in results
2016-05-01 12:16:07.570 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:07.581 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 3000 modifications, 3100 total items in results
2016-05-01 12:16:07.600 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:07.667 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 3100 modifications, 3200 total items in results
2016-05-01 12:16:07.680 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:07.694 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 3200 modifications, 3300 total items in results
2016-05-01 12:16:07.721 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:07.760 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 3300 modifications, 3400 total items in results
2016-05-01 12:16:07.785 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:07.799 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 3400 modifications, 3500 total items in results
2016-05-01 12:16:07.812 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:07.860 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 3500 modifications, 3600 total items in results
2016-05-01 12:16:07.886 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:07.900 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 3600 modifications, 3700 total items in results
2016-05-01 12:16:07.922 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:07.953 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 3700 modifications, 3800 total items in results
2016-05-01 12:16:07.979 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:07.993 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 3800 modifications, 3900 total items in results
2016-05-01 12:16:08.051 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:08.129 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:08.174 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:08.219 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:08.323 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:08.413 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:08.513 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:08.586 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:08.655 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:08.727 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:08.793 RLMIssueSample[12391:5536031] Saved 100 new items in Realm
2016-05-01 12:16:08.793 RLMIssueSample[12391:5536031] Finished saving 5000 total new items in Realm
2016-05-01 12:16:10.367 RLMIssueSample[12391:5535908] 0 deletions, 100 insertions, 3900 modifications, 4000 total items in results
2016-05-01 12:16:10.388 RLMIssueSample[12391:5535908] 0 deletions, 1000 insertions, 4000 modifications, 5000 total items in results

This is great, we import 5000 FeedItems and get notifications for them in acceptable time. But, wait... Stop the app using Xcode. Don't remove it from the device, run it again, and watch the logs again, while it performs a second import. This time they will look like this..

2016-05-01 12:19:00.643 RLMIssueSample[12396:5537024] 5000 items in results at first load
2016-05-01 12:19:00.644 RLMIssueSample[12396:5537024] Staring Import
2016-05-01 12:19:00.763 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:00.821 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:00.867 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:00.937 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:01.047 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:01.123 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:01.177 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:01.231 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:01.289 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:01.378 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:01.418 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:01.458 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:01.540 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:01.606 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:01.648 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:01.740 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:01.823 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:01.886 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:01.955 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:02.007 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:02.064 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:02.138 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:02.182 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:02.227 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:02.315 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:02.358 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:02.398 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:02.497 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:02.569 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:02.615 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:02.662 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:02.762 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:02.810 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:02.854 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:02.946 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:02.994 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:03.086 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:03.141 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:03.194 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:03.239 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:03.321 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:03.374 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:03.422 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:03.460 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:03.509 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:03.564 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:03.654 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:03.702 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:03.759 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:03.862 RLMIssueSample[12396:5537086] Saved 100 new items in Realm
2016-05-01 12:19:03.862 RLMIssueSample[12396:5537086] Finished saving 5000 total new items in Realm

Notice that we only get the initial results, but, no new import notifications. Also, the RLMRealm Notification Listener Thread goes close to 100% CPU usage & stays that way for a while. Here is a screenshot showing this... screen shot 2016-05-01 at 12 24 08 pm

I'm running the app on an iPhone 6 Plus, so your logs/times/results may vary on different devices.

Hopefully you find this useful.

CraigSiemens commented 8 years ago

I just ran into this as well where objects were being reported as modified when I didn't think they should be.

I don't think this is an issue with the implementation, only that the documentation could be a lot clearer. It only says

Collection notifications are delivered ... after each write transaction which changes either any of the objects in the collection, or which objects are in the collection.

There is no mention (that I could find) that if object A has a reference to object B and B is changed, A will be marked as changed as well. I had to create a test project and through trial and error figured out that was the case.

I (and I assume others) made the mistake of thinking that the change notifications would be the same as those reported by NSFetchedResultsController. In reality, it's actually more useful but it can be confusing if you go by the documentation.

Things I think could be improved in the documentation

mrackwitz commented 8 years ago

Thanks a lot @CraigSiemens for your very valuable feedback. I took a stab at addressing the issues in our documentation you pointed out here. As @tgoyne recently landed #3522 which addresses the performance issues for large complex objects graphs, I guess we're done here. I'm looking forward to feedback from y'all once the recent changes are released. As long you're welcome to try it out directly from master while that's recommended only as an option for development. Let us know if you're still experience issues beyond that.

danielgalasko commented 8 years ago

This is a very interesting discussion. In our Application the behaviour of modifying the object at the end of the relationship causing updates is exactly what we want 😄 We have an array of type Element is effectively generic in that we can have ImageElement or FontElement. This is easily modelled using Swift enums but when we persist to Realm we convert the enum to a concrete Realm type. Being able to receive updates when changing the ImageElement on type Element is exactly what we need.

Just adding my voice to the noise, very interesting:)

zh-se commented 6 years ago

So guys, I didn't get how to avoid this behavior.

I have slightly different case, i.e.: There are Feed object, and a FeedItem object. Feed has a to-many relationship items to FeedItem.

When I fetch a new piece of FeedItems and append to Feed's items list it generates a change notification as expected. The notification change has insertions which is also expected. But it has a lot of updates (actually all other items), which is not expected.

I don't understand why the change set has updates (but actuall items wasn't updated) and how to avoid it.

thanks in advance

zh-se commented 6 years ago

The issue was on our side. In our case FeedItem also has a to-one relationship to Feed. When we fetch new items Feed object is changed, and since FeedItem depends on Feed (b/c it has the relationship feed) it leads to update of FeedItem. So that why the notification changeset contains updates.

Thanks

RetVal commented 5 years ago

I think this problem is still not be solved. Why close this issue? And the merge #3522 improved the performances but still bad in this situation!

RetVal commented 5 years ago

And in this situation, I wrap the for-loop, gcd block with autorelease pool, the size of db is growing fast.