Orphan removal during migration to embedded Objects

sirambd commented 2 years ago

How frequently does the bug occur?

Sometimes

Description

We do a migration to embedded objects, and to do this we remove all orphans with the transform function, it works well until we notice that on some devices, we receive this exception At least one object in 'class_Rights' does not have a backlink (data would get lost). Wouldn't the transform function process all files in some cases? Or is the error due to something else?

I've done a lot of tests without ever succeeding in reproducing it, but we do have customers with this problem, although it's a minority.

Here is our migration code:

schema.get(Rights::class.java.simpleName)?.transform { // apply for each right
    val fileId = it.getInt("fileId")
    val file = realm.where(File::class.java.simpleName).equalTo(File::id.name, fileId).findFirst()
    if (file == null) it.deleteFromRealm() // Delete if it's orphan
}?.apply {
    removePrimaryKey()
    removeField("fileId")
    isEmbedded = true
}

A right is orphaned simply if there are no more files associated with it.

Stacktrace & log output

java.lang.IllegalStateException: At least one object in 'class_Rights' does not have a backlink (data would get lost).
    at io.realm.internal.Table.nativeSetEmbedded(Table.java)
    at io.realm.internal.Table.setEmbedded(Table.java:796)
    at io.realm.RealmObjectSchema.setEmbedded(RealmObjectSchema.java:595)
    at com.infomaniak.drive.data.cache.FileMigration.migrate(FileMigration.kt:86)
    at io.realm.BaseRealm$6.onMigrationNeeded(BaseRealm.java:867)
    at io.realm.internal.OsSharedRealm.runMigrationCallback(OsSharedRealm.java:561)
    at io.realm.internal.OsSharedRealm.nativeGetSharedRealm(OsSharedRealm.java)
    at io.realm.internal.OsSharedRealm.<init>(OsSharedRealm.java:175)
    at io.realm.internal.OsSharedRealm.getInstance(OsSharedRealm.java:251)
    at io.realm.BaseRealm.<init>(BaseRealm.java:141)
    at io.realm.BaseRealm.<init>(BaseRealm.java:108)
    at io.realm.Realm.<init>(Realm.java:159)
    at io.realm.Realm.createInstance(Realm.java:495)
    at io.realm.RealmCache.createInstance(RealmCache.java:494)
    at io.realm.RealmCache.doCreateRealmOrGetFromCache(RealmCache.java:461)
    at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:422)
    at io.realm.Realm.getInstance(Realm.java:424)
    at com.infomaniak.drive.data.cache.FileController.getRealmInstance(FileController.kt:312)
    at com.infomaniak.drive.utils.SyncOfflineUtils.startSyncOffline(SyncOfflineUtils.kt:44)
    at com.infomaniak.drive.ui.MainViewModel$syncOfflineFiles$2.invoke(MainViewModel.kt:268)
    at com.infomaniak.drive.ui.MainViewModel$syncOfflineFiles$2.invoke(MainViewModel.kt:267)
    at kotlinx.coroutines.InterruptibleKt.runInterruptibleInExpectedContext(Interruptible.kt:46)
    at kotlinx.coroutines.InterruptibleKt.access$runInterruptibleInExpectedContext(Interruptible.kt:1)
    at kotlinx.coroutines.InterruptibleKt$runInterruptible$2.invokeSuspend(Interruptible.kt:38)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
    at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:39)
    at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
    at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

Can you reproduce the bug?

Not yet

Reproduction Steps

No response

Version

10.8.0

What SDK flavour are you using?

Local Database only

Are you using encryption?

No, not using encryption

Platform OS and version(s)

Android 9

Build environment

Android Studio version: Android Studio Bumblebee | 2021.1.1 Android Build Tools version: ... Gradle version: gradle-7.1.1

cmelchior commented 2 years ago

Hi @sirambd

Yes, the transform method should iterate all objects, but looking at the source code for it, I noticed that we might have a bug there if you delete the object being transformed (as in it might accidentally skip the next object).

I will look more into this and post an update.

sirambd commented 2 years ago

@cmelchior Ok thank you, I will wait for your return

realm / realm-java