error.executeClientReset never completes in case of Flexible Sync

[Santosh-YJS-Micro]

Conditions for the bug:

1. Flexible sync the android App to a specific document in a collection
2. Now quit the android App
3. Delete the document of modify a field in it using Compass/Atlas/Realm-Function/Web-hook
4. Start the app
5. It would lead to a Manual Client Reset situation.
6. First step in Manual Reset is to move the current copy to a backup file ---- achieved by a calling 'error.executeClientReset()'
7. You will find that this call never gets completed.

Reason: Seems like stuck in recursive lock

Code: //-------------------------------------------------------------------------------

private void createSync(){
        // create a Sync configuration with require subscriptions
        syncConfig = new SyncConfiguration.Builder(xsRealmUser.getValue())
                .initialSubscriptions(new SyncConfiguration.InitialFlexibleSyncSubscriptions() {
                    @Override
                    public void configure(Realm realm, MutableSubscriptionSet subscriptions) {
                        Iterator<Subscription> itr= subscriptions.iterator();
                        while(itr.hasNext()){
                            subscriptionList.add(itr.next().getName());
                        }

                        if (!subscriptionList.contains("userSubscription")){
                            subscriptions.add(Subscription.create("userSubscription",
                                    realm.where(xs_user.class)
                                            .equalTo("email","abcd@gmail.com")));
                        }

                    }
                })
                .waitForInitialRemoteData()
                .syncClientResetStrategy(new ManuallyRecoverUnsyncedChangesStrategy() {
                    @Override
                    public void onClientReset(SyncSession session, ClientResetRequiredError error) {
                        Log.i(TAG, "Error from initial subscription: "+error.getMessage());
                        handleManualReset(realmApp,session,error);
                    }
                })
                .build();

         realmAsyncTask = Realm.getInstanceAsync(syncConfig, new Realm.Callback() {
            @Override
            public void onSuccess(Realm realm) {
                backgroundThreadRealm = realm;
                syncReady.postValue(true);
                Log.i(TAG, "onSuccess: sync on success called");
            }

 private void handleManualReset(App app, SyncSession session, ClientResetRequiredError error) {
        if (backgroundThreadRealm !=null && !backgroundThreadRealm.isClosed()){
            backgroundThreadRealm.close();
        }

        try {
            Log.i(TAG, "About to execute the client reset.");
            error.executeClientReset();
            Log.i(TAG, "Executed the client reset.");
        }catch(IllegalStateException e){
            Log.e("EXAMPLE", "Failed to execute the client reset: " + e.getMessage());
    }
 }

//------------------------------------------------------------------------------- The following code never reaches return statement //----------------------------------------------

/**
 * Returns the current number of open Realm instances across all threads in current process that are using this
 * configuration. This includes both dynamic and normal Realms.
 *
 * @param configuration the {@link io.realm.RealmConfiguration} for the Realm.
 * @return number of open Realm instances across all threads.
 */
public static int getGlobalInstanceCount(RealmConfiguration configuration) {
    final AtomicInteger globalCount = new AtomicInteger(0);
    RealmCache.invokeWithGlobalRefCount(configuration, new RealmCache.Callback() {
        @Override
        public void onResult(int count) {
            globalCount.set(count);
        }
    });
    return globalCount.get();
}

//-------------------------------------------------------------

Gets Stuck here: //-----------------------------------------------------------------------

static void invokeWithGlobalRefCount(RealmConfiguration configuration, Callback callback) {
    // NOTE: Although getCache is locked on the cacheMap, this whole method needs to be lock with it as
    // well. Since we need to ensure there is no Realm instance can be opened when this method is called (for
    // deleteRealm).
    // Recursive lock cannot be avoided here.
    synchronized (cachesList) {
        RealmCache cache = getCache(configuration.getPath(), false);
        if (cache == null) {
            callback.onResult(0);
            return;
        }
       ** ## cache.doInvokeWithGlobalRefCount(callback); **
    }
}

//---------------------------------------------------------

[clementetb]

Hi @Santosh-YJS-Micro , is the background realm being closed successfully? It might be accessed from a different thread than where it was created. Realms can only be closed from the same thread they were created.

The background Realm is created on the thread where this method is executed.

realmAsyncTask = Realm.getInstanceAsync(syncConfig, new Realm.Callback() {
            @Override
            public void onSuccess(Realm realm) {
                backgroundThreadRealm = realm;
                syncReady.postValue(true);
                Log.i(TAG, "onSuccess: sync on success called");
            }

but the client reset callback is triggered from the Sync client thread so it would be:

backgroundThreadRealm.close()

Please try closing the Realm from the same thread it was created.

[Santosh-YJS-Micro]

I am afraid this is not the case. The Realm is being closed properly. Due to this same thread issue...to be on the safer side, I have created a SyncManager class to handle Initialization as well as the Reset.

The Manual Reset method used by me is following:

`private void handleManualReset(App app, SyncSession session, ClientResetRequiredError error) {

if (backgroundThreadRealm !=null && !backgroundThreadRealm.isClosed()){
    backgroundThreadRealm.close();
}

try {
    Log.i(TAG, "About to execute the client reset.");  ------ This is logged, so I am sure realm is closed
    error.executeClientReset();
    Log.i(TAG, "Executed the client reset.");----- This is never logged so I am sure executeClientReset has some issue.
}catch(IllegalStateException e){
    Log.e(TAG, "Failed to execute the client reset: " + e.getMessage());

}`

[clementetb]

@Santosh-YJS-Micro we are looking into the issue. Could please you confirm that the Realm gets closed?

At some point, you invoke realm.close() could you add a breakpoint there to confirm that the close method is invoked successfully?

[Santosh-YJS-Micro]

Hi clementetb,

I have already put breakpoint on close(). That's why I submitted the trace in the beginning itself.

If you see the code for handleManualReset(), the execution goes past backgroundThreadRealm.close() and hangs on error.executeClientReset().

Thereafter I entered the execution of error.executeClientReset(). This method calls ---- public static int getGlobalInstanceCount(RealmConfiguration configuration) which is supposed to return globalCount.get().

It was not returned, so I entered this one too.

This invokes RealmCache.invokeWithGlobalRefCount(configuration, new RealmCache.Callback().

Problem is here as this hangs. Whoever has coded it, has also added a comment that // Recursive lock cannot be avoided here.

static void invokeWithGlobalRefCount(RealmConfiguration configuration, Callback callback) {
    // NOTE: Although getCache is locked on the cacheMap, this whole method needs to be lock with it as
    // well. Since we need to ensure there is no Realm instance can be opened when this method is called (for
    // deleteRealm).
    // Recursive lock cannot be avoided here.
    synchronized (cachesList) {
        RealmCache cache = getCache(configuration.getPath(), false);
        if (cache == null) {
            callback.onResult(0);
            return;
        }
       ** ## cache.doInvokeWithGlobalRefCount(callback); **
    }
}

[Santosh-YJS-Micro]

I believe the close() is successful as the execution moved to next instruction.

If you want me to put a breakpoint on close and check something in stack or inner calls of close(); I can do that.

Just let me know what exactly are we trying to find and I will try my best.

[clementetb]

Hi @Santosh-YJS-Micro

There is a deadlock while trying to close the Realm instance within the client reset block. We have opened a PR to address the issue. I don't have any temporal workaround, you might use the snapshot release once it gets merged into releases.

[rorbech]

The fix should already be part of 10.11.1