App crashes then clears library after restoring from backup.

[rajbahakdanyu]

Steps to reproduce

1. Restore backup. tachiyomi_2022-07-03_19-53.proto.gz

Expected behavior

The app should crash and the restored data should be deleted.

Actual behavior

App crashes after a successful restore then when you reopen the application the library will be empty.

Crash logs

App version: 1.5.4 (standard, 75c9b27d8, 92, 2022-05-23T18:09Z) Android version: 12 (SDK 32) Android build ID: SQ3A.220605.009.B1 Device brand: Xiaomi Device manufacturer: Xiaomi Device name: sweet Device model: M2101K6G Device product name: aosp_sweet

--------- beginning of main 07-04 11:10:04.081 28819 2982 E SQLiteLog: (11) database corruption at line 67339 of [5c9a6c0687] 07-04 11:10:04.082 28819 2982 E SQLiteLog: (11) database corruption at line 67489 of [5c9a6c0687] 07-04 11:10:04.082 28819 2982 E SQLiteLog: (11) statement aborts at 22: [SELECT M., COALESCE(MC.category_id, 0) AS category 07-04 11:10:04.082 28819 2982 E SQLiteLog: FROM ( 07-04 11:10:04.082 28819 2982 E SQLiteLog: SELECT mangas., COALESCE(C.unread, 0) AS unread, COALESCE(R.hasread, 0) AS has_read 07-04 11:10:04.082 28819 2982 E SQLiteLog: FROM mangas 07-04 11:10:04.082 28819 2982 E SQLiteLog:
07-04 11:10:04.082 28819 2982 E SupportSQLite: Corruption reported by sqlite on database: /data/user/0/eu.kanade.tachiyomi.j2k/databases/tachiyomi.db 07-04 11:10:04.111 28819 2982 E ACRA : ACRA caught a StorIOException for eu.kanade.tachiyomi.j2k 07-04 11:10:04.111 28819 2982 E ACRA : com.pushtorefresh.storio.StorIOException: Error has occurred during Get operation. query = RawQuery{query=' 07-04 11:10:04.111 28819 2982 E ACRA : SELECT M., COALESCE(MC.category_id, 0) AS category 07-04 11:10:04.111 28819 2982 E ACRA : FROM ( 07-04 11:10:04.111 28819 2982 E ACRA : SELECT mangas., COALESCE(C.unread, 0) AS unread, COALESCE(R.hasread, 0) AS has_read 07-04 11:10:04.111 28819 2982 E ACRA : FROM mangas 07-04 11:10:04.111 28819 2982 E ACRA : LEFT JOIN ( 07-04 11:10:04.111 28819 2982 E ACRA : SELECT manga_id, COUNT() AS unread 07-04 11:10:04.111 28819 2982 E ACRA : FROM chapters 07-04 11:10:04.111 28819 2982 E ACRA : WHERE read = 0 07-04 11:10:04.111 28819 2982 E ACRA : GROUP BY manga_id 07-04 11:10:04.111 28819 2982 E ACRA : ) AS C 07-04 11:10:04.111 28819 2982 E ACRA : ON _id = C.manga_id 07-04 11:10:04.111 28819 2982 E ACRA : LEFT JOIN ( 07-04 11:10:04.111 28819 2982 E ACRA : SELECT manga_id, COUNT() AS hasread 07-04 11:10:04.111 28819 2982 E ACRA : FROM chapters 07-04 11:10:04.111 28819 2982 E ACRA : WHERE read = 1 07-04 11:10:04.111 28819 2982 E ACRA : GROUP BY manga_id 07-04 11:10:04.111 28819 2982 E ACRA : ) AS R 07-04 11:10:04.111 28819 2982 E ACRA : ON _id = R.manga_id 07-04 11:10:04.111 28819 2982 E ACRA : WHERE favorite = 1 07-04 11:10:04.111 28819 2982 E ACRA : GROUP BY _id 07-04 11:10:04.111 28819 2982 E ACRA : ORDER BY title 07-04 11:10:04.111 28819 2982 E ACRA : ) AS M 07-04 11:10:04.111 28819 2982 E ACRA : LEFT JOIN ( 07-04 11:10:04.111 28819 2982 E ACRA : SELECT * FROM mangas_categories) AS MC 07-04 11:10:04.111 28819 2982 E ACRA : ON MC.manga_id = M._id 07-04 11:10:04.111 28819 2982 E ACRA : ', args=[], affectsTables=[], affectsTags=[], observesTables=[mangas_categories, categories, mangas, chapters], observesTags=[]} 07-04 11:10:04.111 28819 2982 E ACRA : at com.pushtorefresh.storio.sqlite.operations.get.PreparedGetListOfObjects.executeAsBlocking(PreparedGetListOfObjects.java:22) 07-04 11:10:04.111 28819 2982 E ACRA : at eu.kanade.tachiyomi.ui.library.LibraryPresenter.access$getLibraryFromDB(LibraryPresenter.kt:7) 07-04 11:10:04.111 28819 2982 E ACRA : at eu.kanade.tachiyomi.ui.library.LibraryPresenter$getLibrary$2$library$1.invokeSuspend(Unknown Source:8) 07-04 11:10:04.111 28819 2982 E ACRA : at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:4) 07-04 11:10:04.111 28819 2982 E ACRA : at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:18) 07-04 11:10:04.111 28819 2982 E ACRA : at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:2) 07-04 11:10:04.111 28819 2982 E ACRA : at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:1) 07-04 11:10:04.111 28819 2982 E ACRA : at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:1) 07-04 11:10:04.111 28819 2982 E ACRA : at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:10) 07-04 11:10:04.111 28819 2982 E ACRA : Suppressed: kotlinx.coroutines.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@e115fc8, Dispatchers.Default] 07-04 11:10:04.111 28819 2982 E ACRA : Caused by: android.database.sqlite.SQLiteDatabaseCorruptException: database disk image is malformed (code 11) 07-04 11:10:04.111 28819 2982 E ACRA : at io.requery.android.database.sqlite.SQLiteConnection.nativeExecuteForCursorWindow(Native Method) 07-04 11:10:04.111 28819 2982 E ACRA : at io.requery.android.database.sqlite.SQLiteConnection.executeForCursorWindow(SQLiteConnection.java:8) 07-04 11:10:04.111 28819 2982 E ACRA : at io.requery.android.database.sqlite.SQLiteSession.executeForCursorWindow(SQLiteSession.java:4) 07-04 11:10:04.111 28819 2982 E ACRA : at io.requery.android.database.sqlite.SQLiteQuery.fillWindow(SQLiteQuery.java:5) 07-04 11:10:04.111 28819 2982 E ACRA : at io.requery.android.database.sqlite.SQLiteCursor.fillWindow(SQLiteCursor.java:4) 07-04 11:10:04.111 28819 2982 E ACRA : at io.requery.android.database.sqlite.SQLiteCursor.getCount(SQLiteCursor.java:2) 07-04 11:10:04.111 28819 2982 E ACRA : at com.pushtorefresh.storio.sqlite.operations.get.PreparedGetListOfObjects.executeAsBlocking(PreparedGetListOfObjects.java:9) 07-04 11:10:04.111 28819 2982 E ACRA : ... 8 more 07-04 11:10:05.599 30954 30954 E e.tachiyomi.j2: Not starting debugger since process cannot load the jdwp agent. 07-04 11:10:06.085 30954 5097 E SQLiteLog: (284) automatic index on C(manga_id) 07-04 11:10:06.085 30954 5097 E SQLiteLog: (284) automatic index on R(manga_id) 07-04 11:10:06.085 30954 5097 E SQLiteLog: (284) automatic index on mangas_categories(manga_id) 07-04 11:10:06.223 5104 5104 E hiyomi.j2k:acr: Not starting debugger since process cannot load the jdwp agent. 07-04 11:10:10.068 30954 5093 E SQLiteLog: (284) automatic index on mangas(_id) 07-04 11:13:43.393 30956 30956 E e.tachiyomi.j2: Not starting debugger since process cannot load the jdwp agent. 07-04 11:13:43.798 30956 5468 E SQLiteLog: (284) automatic index on C(manga_id) 07-04 11:13:43.798 30956 5468 E SQLiteLog: (284) automatic index on R(manga_id) 07-04 11:13:43.799 30956 5468 E SQLiteLog: (284) automatic index on mangas_categories(manga_id)

TachiyomiJ2K version

1.5.4

Android version

Android 12

Device

Redmi Note 10 pro

Other details

No response

Acknowledgements

• [X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open issue.
• [X] I have written a short but informative title.
• [X] If this is an issue with an extension, I should be opening an issue in the extensions repository.
• [X] I am reporting an issue exclusive to this fork. I have also checked that is not an issue on the main version of Tachiyomi
• [X] I have tried the troubleshooting guide.
• [X] I have updated the app to version 1.5.4.
• [X] I have updated all installed extensions.
• [ ] I have admitted that I am a clown by having checked this box, as I have not read these acknowledgements.
• [X] I have filled out all of the requested information in this form.

[Jays2Kings]

I can't reproduce this with the backup provided. Likely with your phone and a large backup, your device is probably killing the restore process. You can try this guide to see if it helps https://dontkillmyapp.com/xiaomi

[Kogar111]

Same library sqlite database corruption problem (since update from Xiaomi.eu android 11 to an android 12.1 custom rom [Derpfest Shinju])

Bug happening on every Tachiyomi fork. Tested with J2K 1.5.4 and 1.5.5 , Standard Tachiyomi 0.13.4 and 0.13.5 and all other current stable versions (SY, AZ)

Backup working on older/other devices, bluestacks and tachidesk. Other apps (novel/manga apps with large library/database too) don't seem to be affected with similar error on that custom rom.

Steps to reproduce Have at least two dozen mangas in library (in multiple categories). Start update of library or even only category of ~20 mangas. After a while, latest at finishing up sqlite corruption happens which deletes the database "Corruption reported by sqlite on database: /data/user/0/eu.kanade.tachiyomi.j2k/databases/tachiyomi.db"

TachiyomiJ2K version 1.5.4 + 1.5.5

Android version Android 12.1 (custom rom: Derpfest Shinju)

Device Redmi K20

Crash logs

App version: 1.5.5 (standard, 6ae512a9b, 93, 2022-07-05T23:03Z) Android version: 12 (SDK 32) Android build ID: SQ3A.220705.003 Device brand: Xiaomi Device manufacturer: Xiaomi Device name: davinci Device model: Redmi K20 Device product name: davinci

--------- beginning of main 07-09 08:00:59.719 25184 1718 E SQLiteLog: (11) database corruption at line 67339 of [5c9a6c0687] 07-09 08:00:59.719 25184 1718 E SQLiteLog: (11) database corruption at line 67489 of [5c9a6c0687] 07-09 08:00:59.719 25184 1718 E SQLiteLog: (11) statement aborts at 51: [SELECT M., COALESCE(MC.category_id, 0) AS category 07-09 08:00:59.719 25184 1718 E SQLiteLog: FROM ( 07-09 08:00:59.719 25184 1718 E SQLiteLog: SELECT mangas., COALESCE(C.unread, 0) AS unread, COALESCE(R.hasread, 0) AS has_read 07-09 08:00:59.719 25184 1718 E SQLiteLog: FROM mangas 07-09 08:00:59.719 25184 1718 E SQLiteLog:
07-09 08:00:59.719 25184 1718 E SupportSQLite: Corruption reported by sqlite on database: /data/user/0/eu.kanade.tachiyomi.j2k/databases/tachiyomi.db 07-09 08:00:59.741 25184 1718 E ACRA : ACRA caught a HttpException for eu.kanade.tachiyomi.j2k 07-09 08:00:59.741 25184 1718 E ACRA : coil.network.HttpException: Error has occurred during Get operation. query = RawQuery{query=' 07-09 08:00:59.741 25184 1718 E ACRA : SELECT M., COALESCE(MC.category_id, 0) AS category 07-09 08:00:59.741 25184 1718 E ACRA : FROM ( 07-09 08:00:59.741 25184 1718 E ACRA : SELECT mangas., COALESCE(C.unread, 0) AS unread, COALESCE(R.hasread, 0) AS has_read 07-09 08:00:59.741 25184 1718 E ACRA : FROM mangas 07-09 08:00:59.741 25184 1718 E ACRA : LEFT JOIN ( 07-09 08:00:59.741 25184 1718 E ACRA : SELECT manga_id, COUNT() AS unread 07-09 08:00:59.741 25184 1718 E ACRA : FROM chapters 07-09 08:00:59.741 25184 1718 E ACRA : WHERE read = 0 07-09 08:00:59.741 25184 1718 E ACRA : GROUP BY manga_id 07-09 08:00:59.741 25184 1718 E ACRA : ) AS C 07-09 08:00:59.741 25184 1718 E ACRA : ON _id = C.manga_id 07-09 08:00:59.741 25184 1718 E ACRA : LEFT JOIN ( 07-09 08:00:59.741 25184 1718 E ACRA : SELECT manga_id, COUNT() AS hasread 07-09 08:00:59.741 25184 1718 E ACRA : FROM chapters 07-09 08:00:59.741 25184 1718 E ACRA : WHERE read = 1 07-09 08:00:59.741 25184 1718 E ACRA : GROUP BY manga_id 07-09 08:00:59.741 25184 1718 E ACRA : ) AS R 07-09 08:00:59.741 25184 1718 E ACRA : ON _id = R.manga_id 07-09 08:00:59.741 25184 1718 E ACRA : WHERE favorite = 1 07-09 08:00:59.741 25184 1718 E ACRA : GROUP BY _id 07-09 08:00:59.741 25184 1718 E ACRA : ORDER BY title 07-09 08:00:59.741 25184 1718 E ACRA : ) AS M 07-09 08:00:59.741 25184 1718 E ACRA : LEFT JOIN ( 07-09 08:00:59.741 25184 1718 E ACRA : SELECT * FROM mangas_categories) AS MC 07-09 08:00:59.741 25184 1718 E ACRA : ON MC.manga_id = M._id 07-09 08:00:59.741 25184 1718 E ACRA : ', args=[], affectsTables=[], affectsTags=[], observesTables=[mangas_categories, categories, mangas, chapters], observesTags=[]} 07-09 08:00:59.741 25184 1718 E ACRA : at com.pushtorefresh.storio.sqlite.operations.get.PreparedGetListOfObjects.executeAsBlocking(PreparedGetListOfObjects.java:22) 07-09 08:00:59.741 25184 1718 E ACRA : at eu.kanade.tachiyomi.ui.library.LibraryPresenter.access$getLibraryFromDB(LibraryPresenter.kt:7) 07-09 08:00:59.741 25184 1718 E ACRA : at eu.kanade.tachiyomi.ui.library.LibraryPresenter$getLibrary$2$library$1.invokeSuspend(Unknown Source:8) 07-09 08:00:59.741 25184 1718 E ACRA : at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:4) 07-09 08:00:59.741 25184 1718 E ACRA : at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:18) 07-09 08:00:59.741 25184 1718 E ACRA : at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:2) 07-09 08:00:59.741 25184 1718 E ACRA : at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:1) 07-09 08:00:59.741 25184 1718 E ACRA : at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:1) 07-09 08:00:59.741 25184 1718 E ACRA : at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:10) 07-09 08:00:59.741 25184 1718 E ACRA : Suppressed: kotlinx.coroutines.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@cb21dd8, Dispatchers.Default] 07-09 08:00:59.741 25184 1718 E ACRA : Caused by: android.database.sqlite.SQLiteDatabaseCorruptException: database disk image is malformed (code 11) 07-09 08:00:59.741 25184 1718 E ACRA : at io.requery.android.database.sqlite.SQLiteConnection.nativeExecuteForCursorWindow(Native Method) 07-09 08:00:59.741 25184 1718 E ACRA : at io.requery.android.database.sqlite.SQLiteConnection.executeForCursorWindow(SQLiteConnection.java:8) 07-09 08:00:59.741 25184 1718 E ACRA : at io.requery.android.database.sqlite.SQLiteSession.executeForCursorWindow(SQLiteSession.java:4) 07-09 08:00:59.741 25184 1718 E ACRA : at io.requery.android.database.sqlite.SQLiteQuery.fillWindow(SQLiteQuery.java:5) 07-09 08:00:59.741 25184 1718 E ACRA : at io.requery.android.database.sqlite.SQLiteCursor.fillWindow(SQLiteCursor.java:4) 07-09 08:00:59.741 25184 1718 E ACRA : at io.requery.android.database.sqlite.SQLiteCursor.getCount(SQLiteCursor.java:2) 07-09 08:00:59.741 25184 1718 E ACRA : at com.pushtorefresh.storio.sqlite.operations.get.PreparedGetListOfObjects.executeAsBlocking(PreparedGetListOfObjects.java:9) 07-09 08:00:59.741 25184 1718 E ACRA : ... 8 more 07-09 08:01:02.408 31423 31423 E e.tachiyomi.j2k: Not starting debugger since process cannot load the jdwp agent. 07-09 08:01:03.435 31423 5734 E SQLiteLog: (284) automatic index on C(manga_id) 07-09 08:01:03.435 31423 5734 E SQLiteLog: (284) automatic index on R(manga_id) 07-09 08:01:03.435 31423 5734 E SQLiteLog: (284) automatic index on mangas_categories(manga_id) 07-09 08:01:03.538 31423 5734 E SQLiteLog: (284) automatic index on mangas(_id) 07-09 08:01:03.634 5749 5749 E hiyomi.j2k:acra: Not starting debugger since process cannot load the jdwp agent.

Edit 2022.07.13: Just connected my phone to android studio and am seeing these logs. A lot of

2022-07-13 11:50:56.437 2083-2324/? D/PeopleSpaceWidgetMgr: Sbn doesn't contain valid PeopleTileKey: null/0/eu.kanade.tachiyomi.sy

Maybe some other interesting log lines

2022-07-13 11:50:39.825 31214-8318/? I/de.tachiyomi.sy: Background young concurrent copying GC freed 1273336(64MB) AllocSpace objects, 5(10220KB) LOS objects, 53% free, 63MB/136MB, paused 57us,30us total 107.491ms

2022-07-13 11:50:54.974 1556-5279/? E/NotificationService: Package enqueue rate is 16.150972. Shedding 0|eu.kanade.tachiyomi.sy|-503|null|10572. package=eu.kanade.tachiyomi.sy

2022-07-13 11:50:56.912 31214-8318/? I/de.tachiyomi.sy: Background concurrent copying GC freed 2807602(95MB) AllocSpace objects, 14(10MB) LOS objects, 72% free, 35MB/131MB, paused 53us,34us total 130.752ms

2022-07-13 11:50:57.279 2083-2083/? W/NotifBindPipeline: Row is not set so pipeline will not run. notif = 0|eu.kanade.tachiyomi.sy|-504|null|10572

And at last the library/database gets wiped because there is some corrupted SQLite message.

2022-07-13 11:54:58.168 31214-14631/? E/SupportSQLite: Corruption reported by sqlite on database: /data/user/0/eu.kanade.tachiyomi.sy/databases/tachiyomi.db 2022-07-13 11:54:58.178 31214-14631/? W/SupportSQLite: deleting the database file: /data/user/0/eu.kanade.tachiyomi.sy/databases/tachiyomi.db

Just before the wipe, because of the corrupted SQLite message, everything in the library is still visible and seems selectable The same happens with all other Tachiyomi forks

Looked at Tachiyomi-Preview Maybe that log line gets omitted in stable fork versions. Afterwards Tachiyomi-Preview crashed and lost the library.

2022-07-13 13:15:56.801 23825-12346/? E/ACRA: ACRA caught a NullPointerException for eu.kanade.tachiyomi.debug java.lang.NullPointerException at eu.kanade.tachiyomi.app.MangasQueriesImpl$getRecentlyUpdated$1.invoke(DatabaseImpl.kt:3) at eu.kanade.tachiyomi.app.MangasQueriesImpl$getRecentlyUpdated$1.invoke(DatabaseImpl.kt:1) at com.squareup.sqldelight.Query.executeAsList(Query.kt:3) at com.squareup.sqldelight.runtime.coroutines.FlowQuery$mapToList$1$1.invoke(FlowExtensions.kt:5) at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:1) at kotlinx.coroutines.BuildersKt__Builders_commonKt.withContext(Builders.common.kt:9) at kotlinx.coroutines.BuildersKt.withContext(Unknown Source:1) at com.squareup.sqldelight.runtime.coroutines.FlowQuery$mapToList$$inlined$map$1$2.emit(Collect.kt:5) at kotlinx.coroutines.flow.internal.SafeCollectorKt$emitFun$1.invoke(SafeCollector.kt:2) at kotlinx.coroutines.flow.internal.SafeCollectorKt$emitFun$1.invoke(SafeCollector.kt:1) at kotlinx.coroutines.flow.internal.SafeCollector.emit(SafeCollector.kt:17) at kotlinx.coroutines.flow.internal.SafeCollector.emit(SafeCollector.kt:1) at com.squareup.sqldelight.runtime.coroutines.FlowQuery$asFlow$1.invokeSuspend(FlowExtensions.kt:9) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:4) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:18) at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:2) at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:1) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:1) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:10) Suppressed: kotlinx.coroutines.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@e748755, Dispatchers.IO]

Edit2 2022.07.16: Second Rom(PixelOS - 13 July 2022) seems to have the same bug. I think Tachiyomi can not handle optimisations of the custom rom. Going through the changelog of PixelOS i found that point "Configure SQLite to operate in MEMORY mode". Maybe SQLite database is just getting disconnected on request end, but since memory mode is used data is not persistently saved or data is thrashed if library is too large for memory. Third rom (ArrowOS 12.1 - 2022-07-01) does not seem to have that problem, but is unfortunately a much more basic rom.

Fourth rom (CherishOs [Cherish-OS-v3.9-20220712-1708-davinci-UNOFFICIAL-GApps] has the bug again. But that rom seems to use "Configure SQLite to operate in MEMORY mode" since version CherishOS v2.5 https://forum.xda-developers.com/t/rom-11-0-r-cherishos-v2-5-official-whyred-25-05-2020.4282069/

Anyone else on custom roms has trouble with large library? Sometimes even from smaller library (100-300) depending on how many chapters each manga got

[Kogar111]

Seems like a Pragma fix is needed to force database to be persistent on over optimised custom roms.

In following dev/preview versions i have no database corruption anymore

https://github.com/jobobby04/TachiyomiSYPreview/releases/tag/464

https://github.com/tachiyomiorg/tachiyomi-preview/releases/tag/r4687