Invalid Value for Primary Key

[nanoxd]

I keep running into errors when attempting to call updateFetchRequest. It works fine when it is called once, but while switching through my data's different states via predicates, it gives me the following stack trace:

2015-05-28 21:27:45.534 app[92238:28255111] *** Terminating app due to uncaught exception 'RLMException', reason: 'Invalid value '96356644' for primary key'
*** First throw call stack:
(
    0   CoreFoundation                      0x000000010ee42c65 __exceptionPreprocess + 165
    1   libobjc.A.dylib                     0x0000000111929bb7 objc_exception_throw + 45
    2   Realm                               0x000000010e576385 RLMGetObject + 1093
    3   Realm                               0x000000010e5640d8 +[RLMObject objectInRealm:forPrimaryKey:] + 136
    4   RBQFetchedResultsController         0x000000010e5103c0 -[RBQFetchedResultsController createChangeSetsWithAddedSafeObjects:deletedSafeObjects:changedSafeObjects:state:] + 2352
    5   RBQFetchedResultsController         0x000000010e50c07d -[RBQFetchedResultsController calculateChangesWithAddedSafeObjects:deletedSafeObjects:changedSafeObjects:realm:] + 2173
    6   RBQFetchedResultsController         0x000000010e50aed8 __58-[RBQFetchedResultsController registerChangeNotifications]_block_invoke + 1208
    7   RBQFetchedResultsController         0x000000010e51d406 -[RBQRealmNotificationManager sendNotificationsWithRealm:entityChanges:] + 518
    8   RBQFetchedResultsController         0x000000010e51f660 __50-[RBQRealmChangeLogger registerChangeNotification]_block_invoke + 256
    9   Realm                               0x000000010e5bd58a -[RLMRealm sendNotifications:] + 986
    10  Realm                               0x000000010e5bd9c6 -[RLMRealm commitWriteTransaction] + 262
    11  app                          0x000000010e092a3c _TZFC105pp5Shift13fromJSONArrayfMS0_FPSs9AnyObject_GSaS0__ + 1964
    12  app                          0x000000010e05dc3b _TFFC10app17CalendarViewModel15invokeShiftListFS0_FTGSqGVSs10DictionarySSPSs9AnyObject___10completionFT_T__T_U_FTGSQCSo22AFHTTPRequestOperation_GSQPS2____T_ + 331
    13  app                          0x000000010df8fdb3 _TTRXFo_oGSQCSo22AFHTTPRequestOperation_oGSQPSs9AnyObject___dT__XFo_iTGSQS__GSQPS0_____iT__ + 35
    14  app                          0x000000010e05bbd1 _TPA__TTRXFo_oGSQCSo22AFHTTPRequestOperation_oGSQPSs9AnyObject___dT__XFo_iTGSQS__GSQPS0_____iT__ + 81
    15  app                          0x000000010e02a097 _TTRXFo_iTGSQCSo22AFHTTPRequestOperation_GSQPSs9AnyObject____iT__XFo_oGSQS__oGSQPS0____dT__ + 39
    16  app                          0x000000010e02a0e9 _TTRXFo_oGSQCSo22AFHTTPRequestOperation_oGSQPSs9AnyObject___dT__XFdCb_dGSQS__dGSQPS0____dT__ + 73
    17  SBJSONRPCClient                     0x000000010e858067 __67-[SBJSONRPCClient HTTPRequestOperationWithRequest:success:failure:]_block_invoke + 471
    18  AFNetworking                        0x000000010e3f6b48 __64-[AFHTTPRequestOperation setCompletionBlockWithSuccess:failure:]_block_invoke46 + 40
    19  libdispatch.dylib                   0x0000000111c80186 _dispatch_call_block_and_release + 12
    20  libdispatch.dylib                   0x0000000111c9f614 _dispatch_client_callout + 8
    21  libdispatch.dylib                   0x0000000111c87a1c _dispatch_main_queue_callback_4CF + 1664
    22  CoreFoundation                      0x000000010edaa1f9 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 9
    23  CoreFoundation                      0x000000010ed6bdcb __CFRunLoopRun + 2043
    24  CoreFoundation                      0x000000010ed6b366 CFRunLoopRunSpecific + 470
    25  GraphicsServices                    0x0000000114333a3e GSEventRunModal + 161
    26  UIKit                               0x000000010ffaf900 UIApplicationMain + 1282
    27  app                          0x000000010e057d27 main + 135
    28  libdyld.dylib                       0x0000000111cd3145 start + 1
)

I can successfully switch once to a different fetchRequest without crashing. Any more times crashes the app. It also doesn't happen on the same primary key, it's random.

[bigfish24]

Which version of Realm are you running?

[nanoxd]

0.93.0

  - RBQFetchedResultsController (1.8.7):
    - Realm
  - Realm (0.93.0):
    - Realm/Headers (= 0.93.0)
  - Realm/Headers (0.93.0)

Method I'm using for calling this:

func performUpdatedFetch(filtered: Bool, isCovered: Bool) {
    var fetchRequest = RBQFetchRequest(entityName: Shift.className(), inRealm: realm)
    let shiftDateSortDescriptor = RLMSortDescriptor(property: "startDate", ascending: true)
    let sectionNameSortDescriptor = RLMSortDescriptor(property: "sectionName", ascending: true)
    fetchRequest.sortDescriptors = [shiftDateSortDescriptor, sectionNameSortDescriptor]

    if filtered {
      let predicate = NSPredicate(format: "isCovered = %@", isCovered)
      fetchRequest.predicate = predicate
    }

    fetchedResultsController.updateFetchRequest(fetchRequest, sectionNameKeyPath: "sectionName", andPeformFetch: true)
    calendarTableView.reloadData()
  }

[bigfish24]

I have a sneaking suspicion this might be related to this issue: https://github.com/realm/realm-cocoa/issues/2018

Might try reverting to 0.92.3 (or was it .4) and seeing if problem still occurs.

[nanoxd]

Attempted backtracking through Realm versions.

Version	Error on primary key
0.92.4	Yes
0.92.3	Yes
0.92.2	Yes
0.92.0	Yes
0.91.5	No, error on //Pods/Realm/Realm/RLMObject_Private.hpp:21:9: 'realm/link_view.hpp' file not found
0.91.4	Yes
0.91.3	Yes

[bigfish24]

Sorry I misread your post, the error is happening on all versions, right?

[nanoxd]

Yes, that is correct.

[bigfish24]

Oh, I think I know what is going on. The change notification from a different thread is happening while the update of the fetch request is occurring on the main thread.

[bigfish24]

I think I have an easy fix for this.

[nanoxd]

That would make sense. I'm parsing in JSON objects in a background thread on the ViewModel but performing the fetch updates on the main queue. The only reason why I ruled that out before is that it works once without crashing.

[bigfish24]

Yeah it is a timing issue, the two have to overlap in the right way to cause the issue, thus it is intermittent.

[bigfish24]

Ok pushed a potential fix on this branch: https://github.com/Roobiq/RBQFetchedResultsController/tree/updateFetchFix

You should probably dispatch the updateFetchRequest call because it is possible that background change processing is occurring and the updateFetchRequest will wait until that finishes to perform the fetch after the update. You can then dispatch_async back to the main thread to do the tableView reloadData call.

[bigfish24]

Went from this:

- (void)updateFetchRequest:(RBQFetchRequest *)fetchRequest
        sectionNameKeyPath:(NSString *)sectionNameKeyPath
            andPeformFetch:(BOOL)performFetch
{
    // Updating the fetch request will force rebuild of cache automatically
    _sectionNameKeyPath = sectionNameKeyPath;
    _fetchRequest = fetchRequest;

    if (performFetch) {
        [self performFetch];
    }
}

to this:

- (void)updateFetchRequest:(RBQFetchRequest *)fetchRequest
        sectionNameKeyPath:(NSString *)sectionNameKeyPath
            andPeformFetch:(BOOL)performFetch
{
    // Turn off change notifications since we are replacing fetch request
    // Change notifications will be re-registered if performFetch is called
    [self unregisterChangeNotifications];

    // Updating the fetch request will force rebuild of cache automatically
    _sectionNameKeyPath = sectionNameKeyPath;
    _fetchRequest = fetchRequest;

    if (performFetch) {
        // Only performFetch if the change processing is finished
        @synchronized(self) {
            [self performFetch];
        }
    }
}

[nanoxd]

Trying it now.

[nanoxd]

In some cases, I get the old error. In others I get

2015-05-28 23:02:13.480 app[13125:28508587] *** Terminating app due to uncaught exception 'RLMException', reason: 'Object has been deleted or invalidated.'
*** First throw call stack:
(
    0   CoreFoundation                      0x000000010df6cc65 __exceptionPreprocess + 165
    1   libobjc.A.dylib                     0x0000000110a53bb7 objc_exception_throw + 45
    2   Realm                               0x000000010d6765b5 _ZL17RLMVerifyAttachedP13RLMObjectBase + 85
    3   Realm                               0x000000010d6766e5 _ZL27RLMVerifyInWriteTransactionP13RLMObjectBase + 21
    4   Realm                               0x000000010d6734bd _ZL11RLMSetValueP13RLMObjectBasemx + 29
    5   Realm                               0x000000010d679467 ___ZL13RLMMakeSetterIxxEPFvvEmb_block_invoke_2 + 55
    6   RBQFetchedResultsController         0x000000010d635445 __58-[RBQFetchedResultsController registerChangeNotifications]_block_invoke_2 + 245
    7   RBQFetchedResultsController         0x000000010d635517 __58-[RBQFetchedResultsController registerChangeNotifications]_block_invoke374 + 39
    8   libdispatch.dylib                   0x0000000110daa186 _dispatch_call_block_and_release + 12
    9   libdispatch.dylib                   0x0000000110dc9614 _dispatch_client_callout + 8
    10  libdispatch.dylib                   0x0000000110db1a1c _dispatch_main_queue_callback_4CF + 1664
    11  CoreFoundation                      0x000000010ded41f9 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 9
    12  CoreFoundation                      0x000000010de95dcb __CFRunLoopRun + 2043
    13  CoreFoundation                      0x000000010de95366 CFRunLoopRunSpecific + 470
    14  GraphicsServices                    0x000000011345da3e GSEventRunModal + 161
    15  UIKit                               0x000000010f0d9900 UIApplicationMain + 1282
    16  app                          0x000000010d181c87 main + 135
    17  libdyld.dylib                       0x0000000110dfd145 start + 1
)

[nanoxd]

Code to perform fetch

  func performUpdatedFetch(filtered: Bool, isCovered: Bool) {
    var fetchRequest = RBQFetchRequest(entityName: Shift.className(), inRealm: realm)
    let shiftDateSortDescriptor = RLMSortDescriptor(property: "startDate", ascending: true)
    let sectionNameSortDescriptor = RLMSortDescriptor(property: "sectionName", ascending: true)
    fetchRequest.sortDescriptors = [shiftDateSortDescriptor, sectionNameSortDescriptor]

    if filtered {
      let predicate = NSPredicate(format: "isCovered = %@", isCovered)
      fetchRequest.predicate = predicate
    }

    dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), {
      self.fetchedResultsController.updateFetchRequest(fetchRequest, sectionNameKeyPath: "sectionName", andPeformFetch: true)

      dispatch_async(dispatch_get_main_queue(), {
        self.calendarTableView.reloadData()
      })
    })
  }

[bigfish24]

Ok, I moved the @synchronized directive to include the unregistering of the notification too:

- (void)updateFetchRequest:(RBQFetchRequest *)fetchRequest
        sectionNameKeyPath:(NSString *)sectionNameKeyPath
            andPeformFetch:(BOOL)performFetch
{
    @synchronized(self) {
        // Turn off change notifications since we are replacing fetch request
        // Change notifications will be re-registered if performFetch is called
        [self unregisterChangeNotifications];

        // Updating the fetch request will force rebuild of cache automatically
        _sectionNameKeyPath = sectionNameKeyPath;
        _fetchRequest = fetchRequest;

        if (performFetch) {
            // Only performFetch if the change processing is finished
            [self performFetch];
        }
    }
}

update pushed to the branch

[nanoxd]

No change, similar stack trace as before. Another interesting thing I've noticed is when this occurs, Realm says there are 1000 items in the table.

[nanoxd]

I figured out a fix. I'm not sure how much this has to do with the code you added but if I change my primary key to be a string instead of an Int, no more crashes.

[bigfish24]

Interesting... is it possible for you to enable "All Exceptions" breakpoint and give me the line numbers (both old and new error) of the crash when using Int?

[bigfish24]

I think I understand what is going on with the second error, and pushed a fix. But the first error, still has me confused.

[bigfish24]

The exception for the first error comes in this function:

id RLMGetObject(RLMRealm *realm, NSString *objectClassName, id key) {
    RLMCheckThread(realm);

    RLMObjectSchema *objectSchema = realm.schema[objectClassName];

    RLMProperty *primaryProperty = objectSchema.primaryKeyProperty;
    if (!primaryProperty) {
        NSString *msg = [NSString stringWithFormat:@"%@ does not have a primary key", objectClassName];
        @throw RLMException(msg);
    }

    if (!objectSchema.table) {
        // read-only realms may be missing tables since we can't add any
        // missing ones on init
        return nil;
    }

    size_t row = realm::not_found;
    if (primaryProperty.type == RLMPropertyTypeString) {
        if (NSString *str = RLMDynamicCast<NSString>(key)) {
            row = objectSchema.table->find_first_string(primaryProperty.column, RLMStringDataWithNSString(str));
        }
        else {
            @throw RLMException([NSString stringWithFormat:@"Invalid value '%@' for primary key", key]);
        }
    }
    else {
        if (NSNumber *number = RLMDynamicCast<NSNumber>(key)) {
            row = objectSchema.table->find_first_int(primaryProperty.column, number.longLongValue);
        }
        else {
            @throw RLMException([NSString stringWithFormat:@"Invalid value '%@' for primary key", key]);
        }
    }

    if (row == realm::not_found) {
        return nil;
    }

    return RLMCreateObjectAccessor(realm, objectSchema, row);
}

specifically this line:

if (NSNumber *number = RLMDynamicCast<NSNumber>(key)) {

So it would be helpful to know what is the class of key in the debugger.

[nanoxd]

key __NSCFNumber *  (long)71410696
str NSString *  nil
primaryProperty RLMProperty *   0x7fb0b0207df0  0x00007fb0b0207df0
row size_t  18446744073709551615    18446744073709551615
objectSchema    RLMObjectSchema *   0x7fb0b0207d30  0x00007fb0b0207d30
not_found   size_t  18446744073709551615    18446744073709551615

crashes on

id RLMGetObject(RLMRealm *realm, NSString *objectClassName, id key) {
    RLMCheckThread(realm);

    RLMObjectSchema *objectSchema = realm.schema[objectClassName];

    RLMProperty *primaryProperty = objectSchema.primaryKeyProperty;
    if (!primaryProperty) {
        NSString *msg = [NSString stringWithFormat:@"%@ does not have a primary key", objectClassName];
        @throw RLMException(msg);
    }

    if (!objectSchema.table) {
        // read-only realms may be missing tables since we can't add any
        // missing ones on init
        return nil;
    }

    size_t row = realm::not_found;
    if (primaryProperty.type == RLMPropertyTypeString) {
        if (NSString *str = RLMDynamicCast<NSString>(key)) {
            row = objectSchema.table->find_first_string(primaryProperty.column, RLMStringDataWithNSString(str));
        }
        else {
>            @throw RLMException([NSString stringWithFormat:@"Invalid value '%@' for primary key", key]);
        }
    }
    else {
        if (NSNumber *number = RLMDynamicCast<NSNumber>(key)) {
            row = objectSchema.table->find_first_int(primaryProperty.column, number.longLongValue);
        }
        else {
            @throw RLMException([NSString stringWithFormat:@"Invalid value '%@' for primary key", key]);
        }
    }

    if (row == realm::not_found) {
        return nil;
    }

    return RLMCreateObjectAccessor(realm, objectSchema, row);
}

[bigfish24]

Can you verify if you delete the Realm file in between your tests on older versions of Realm earlier in this discussion? If you didn't can you use the master branch of RBQFRC, revert to Realm 0.92.4 and delete the Realm file (deleting the app and rebuilding will work), and see if the problem still occurs?

[nanoxd]

Yes, I deleted the file prior to encountering the bug. The switching between realm versions resulted in a 650mb file

[bigfish24]

No, I mean did you delete the Realm file in between each subsequent downgrade of Realm?

[bigfish24]

I missed the ">" in your post (not enough coffee yet!) and the problem seems to be Realm treated your primary key property as a String, since:

if (primaryProperty.type == RLMPropertyTypeString) {

resolved YES. This appears to be related to int primary key indexing issues that appeared in Realm 0.93.0.

[nanoxd]

Sorry for the delay in response. Yes, for some reason it did. I think the issue you linked before was the key to all of this in the first place. Although the updates you made were significant in avoiding concurrency issues. realm/realm-cocoa#2024 is supposed to fix this

Thanks! I owe you a beer 😀

[bigfish24]

Awesome, well I will merge in the concurrency improvements to master. Glad all is working!

[matrixx]

Hi, I have this same problem with version 0.95.0. Could there be something still missing? It seems to go away when I change my primary key type from NSInteger to NSString.

[bigfish24]

Which version are you running of FRC?

[matrixx]

2.0

[bigfish24]

Try updating to 2.1 and let me know if it still persists.

[matrixx]

Wow, it works in 2.1, thanks! Didn't notice there was update available, and I would have been probably settled for the string primary key without your fast response :)

[bigfish24]

Glad it is working!