Concurrency issue [reproduced through tests]

[idcuesta]

Description

I am currently using Encrypted Core Data in a project, and I am experiencing a concurrency issue.

My project has a Core Data Stack like this one

• Main Queue Context hangs from a Private Context, so when save operations are made it doesn't freeze main thread when persisting data into the database. Saves from this context come from direct user interaction.
• Private Queue Context hangs from the same Private Context, and it is used to process batch updates into the database, from updates models that come from network requests. It is a sibling to Main Queue Context and it is listening to his update notifications to merge changes.
• In my UIViewController I am using a NSFetchedResultsController that is fetching objects from the same entities than the ones used by the batch update operation happening in the background. It is using the Main Queue Context.

Under this scenario I get random deadlocks when a performFetch: from my NSFetchedResultsController happens at the same time the Private Queue Context is reading/writing from the same entities.

I have also tried to follow recommendations from Core Data Performance Optimization and Debugging video from WWDC 2013 and make my Private Queue Context and Main Queue Context use their own Persistence Store Coordinators, but when doing that I constantly get this error from save operations Error Domain=NSSQLiteErrorDomain Code=5 "(null)" UserInfo={EncryptedStoreErrorMessage=database is locked}.

When using a default store (non encrypted) there are no deadlocks and everything works fine.

I have added some unit tests that demonstrate the 3 different behaviors for each Core Data Stack. Different Core Data Stacks are running exactly the same tests and the results are:

Tests for the default SQLite Store

• SingleDefaultStoreManager defines the Core Data Stack described above by the image, and it uses a Store of type NSSQLiteStoreType with journal mode disabled, trying to simulate as much as possible the way EncryptedStore works.
• testConcurrentInsertOperationsOnDefaultStore and testConcurrentUpdateOperationsOnDefaultStore are passing consistently

  Tests for the Encrypted Store

• SingleEncryptedStoreManager defines the Core Data Stack described above by the image, and it uses an Encrypted Store.
• testConcurrentInsertOperationsOnEncryptedStore fails most of the times (~80%) with error waitForExpectationsWithTimeout: timed out but was unable to run the timeout handler because the main thread was unresponsive (0.5 seconds is allowed after the wait times out). Conditions that may cause this include processing blocking IO on the main thread, calls to sleep(), deadlocks, and synchronous IPC. Xcode will attempt to relaunch the process and continue with the next test...
• testConcurrentUpdateOperationsOnEncryptedStore fails all the times with the same error

  Tests for the Double Encrypted Store

• DoubleEncryptedStoreManager defines the Core Data Stack, where Main Queue Context and Private Queue Context use different Encrypted Stores that point to the same encrypted database.
• testConcurrentInsertOperationsOnDoubleEncryptedStore and testConcurrentUpdateOperationsOnDoubleEncryptedStore are failing consistently, because of error Error Domain=NSSQLiteErrorDomain Code=5 "(null)" UserInfo={EncryptedStoreErrorMessage=database is locked}

[ppaulojr]

This unit test is designed to fail current version. So the fact it fails is a very good sign to merge it!

[ppaulojr]

Hey folks, if you could merge this one that would be super helpful to solve this ugly deadlock problem.

[DanielBroad]

I understand this doesn't fix any issue, it just introduces a failing test?

[ppaulojr]

Yes, that's the idea of this PR. We are noticing this deadlock problem and the tests are an easy way to reproduce it and assist the process of fixing it.

In the meantime I'm investigating the problem in EncryptedCoreData source so I can make a PR fixing it too.

[idcuesta]

We finally migrated to a different encryption solution (Realm).

I am closing this as it seems it is not useful.