search

MuntashirAkon / AppManager

A full-featured package manager and viewer for Android
https://muntashirakon.github.io/AppManager/
Other
4.9k stars 277 forks source link

Backing up external data is corrupted #473

Closed Cyberavater closed 2 years ago

Cyberavater commented 3 years ago

App Name: Mobile Legends AM version: 1322 Backup Volume: ExtSDCard Log:

E/BackupOp: Failed to backup data directory at /storage/emulated/0/Android/data/com.mobile.legends
io.github.muntashirakon.AppManager.backup.BackupException: Failed to backup data directory at /storage/emulated/0/Android/data/com.mobile.legends
    at io.github.muntashirakon.AppManager.backup.BackupOp.backupData(BackupOp.java:258)
    at io.github.muntashirakon.AppManager.backup.BackupOp.runBackup(BackupOp.java:163)
    at io.github.muntashirakon.AppManager.backup.BackupManager.backup(BackupManager.java:90)
    at io.github.muntashirakon.AppManager.batchops.BatchOpsManager.lambda$opBackupRestore$0$BatchOpsManager(BatchOpsManager.java:269)
    at io.github.muntashirakon.AppManager.batchops.-$$Lambda$BatchOpsManager$fRbL-cOU1e30grqu_qUYfkugpEc.run(Unknown Source:10)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:462)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
    at java.lang.Thread.run(Thread.java:919)
Caused by: java.io.IOException: java.io.FileNotFoundException: /storage/C97F-1415/AppManager/.tmp/backup_1623079911633/data2.tar.gz.0: open failed: EACCES (Permission denied)
    at io.github.muntashirakon.io.SplitOutputStream.checkCurrentStream(SplitOutputStream.java:97)
    at io.github.muntashirakon.io.SplitOutputStream.write(SplitOutputStream.java:65)
    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
    at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
    at org.apache.commons.compress.compressors.gzip.GzipCompressorOutputStream.close(GzipCompressorOutputStream.java:196)
    at io.github.muntashirakon.AppManager.utils.TarUtils.create(TarUtils.java:117)
    at io.github.muntashirakon.AppManager.backup.BackupOp.backupData(BackupOp.java:254)
    ... 9 more
    Suppressed: java.io.IOException: java.io.FileNotFoundException: /storage/C97F-1415/AppManager/.tmp/backup_1623079911633/data2.tar.gz.0: open failed: EACCES (Permission denied)
        at io.github.muntashirakon.io.SplitOutputStream.checkCurrentStream(SplitOutputStream.java:97)
        at io.github.muntashirakon.io.SplitOutputStream.write(SplitOutputStream.java:65)
        at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
        at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
        at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
        at io.github.muntashirakon.AppManager.utils.TarUtils.create(TarUtils.java:78)
        ... 10 more
    Caused by: java.io.FileNotFoundException: /storage/C97F-1415/AppManager/.tmp/backup_1623079911633/data2.tar.gz.0: open failed: EACCES (Permission denied)
        at libcore.io.IoBridge.open(IoBridge.java:496)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:235)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:186)
        at io.github.muntashirakon.io.ProxyOutputStream.<init>(ProxyOutputStream.java:31)
        at io.github.muntashirakon.io.SplitOutputStream.checkCurrentStream(SplitOutputStream.java:93)
        ... 15 more
    Caused by: android.system.ErrnoException: open failed: EACCES (Permission denied)
        at libcore.io.Linux.open(Native Method)
        at libcore.io.ForwardingOs.open(ForwardingOs.java:167)
        at libcore.io.BlockGuardOs.open(BlockGuardOs.java:252)
        at libcore.io.ForwardingOs.open(ForwardingOs.java:167)
        at android.app.ActivityThread$AndroidOs.open(ActivityThread.java:7256)
        at libcore.io.IoBridge.open(IoBridge.java:482)
        ... 19 more
Caused by: java.io.FileNotFoundException: /storage/C97F-1415/AppManager/.tmp/backup_1623079911633/data2.tar.gz.0: open failed: EACCES (Permission denied)
    at libcore.io.IoBridge.open(IoBridge.java:496)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:235)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:186)
    at io.github.muntashirakon.io.ProxyOutputStream.<init>(ProxyOutputStream.java:31)
    at io.github.muntashirakon.io.SplitOutputStream.checkCurrentStream(SplitOutputStream.java:93)
    ... 16 more
Caused by: android.system.ErrnoException: open failed: EACCES (Permission denied)
    at libcore.io.Linux.open(Native Method)
    at libcore.io.ForwardingOs.open(ForwardingOs.java:167)
    at libcore.io.BlockGuardOs.open(BlockGuardOs.java:252)
    at libcore.io.ForwardingOs.open(ForwardingOs.java:167)
    at android.app.ActivityThread$AndroidOs.open(ActivityThread.java:7256)
    at libcore.io.IoBridge.open(IoBridge.java:482)
    ... 20 more
MuntashirAkon commented 3 years ago

How many times have you tried to back up the app? The issue looks odd. It implied that the privileged server wasn't running and used the fallback method (which would fail for external SD Card until #115 is implemented).

Cyberavater commented 3 years ago

Like 3 or 4 times. It's really specific to this app. There was another issue with another app which was fixed after reporting via TG. Anyways, I need more time to test it thoroughly, i.e try to backup this app in device storage and such.

MuntashirAkon commented 3 years ago

Could you try again with the latest debug? There's a slight chance that App Manager will display false-positive results, be sure to check if the files are actually present after the backup is finished.

Cyberavater commented 3 years ago

At the time opened this issue, ONLY this app wasn't backing up. Now, with the latest version I can't restore ANY app, thus can't conduct any tests. log:

E/Users: Could not get list of users
java.lang.SecurityException: You either need MANAGE_USERS or CREATE_USERS permission to: query users
    at android.os.Parcel.createException(Parcel.java:2071)
    at android.os.Parcel.readException(Parcel.java:2039)
    at android.os.Parcel.readException(Parcel.java:1987)
    at android.os.IUserManager$Stub$Proxy.getUsers(IUserManager.java:1747)
    at io.github.muntashirakon.AppManager.users.Users.getAllUsers(Users.java:72)
    at io.github.muntashirakon.AppManager.users.Users.getUsersIds(Users.java:101)
    at io.github.muntashirakon.AppManager.backup.BackupFlags$1.<init>(BackupFlags.java:225)
    at io.github.muntashirakon.AppManager.backup.BackupFlags.getBackupFlagsMap(BackupFlags.java:202)
    at io.github.muntashirakon.AppManager.backup.BackupFlags.getFormattedFlagNames(BackupFlags.java:69)
    at io.github.muntashirakon.AppManager.backup.BackupDialogFragment.onCreateDialog(BackupDialogFragment.java:143)
    at androidx.fragment.app.DialogFragment.prepareDialog(DialogFragment.java:644)
    at androidx.fragment.app.DialogFragment.onGetLayoutInflater(DialogFragment.java:558)
    at androidx.fragment.app.Fragment.performGetLayoutInflater(Fragment.java:1654)
    at androidx.fragment.app.FragmentStateManager.createView(FragmentStateManager.java:493)
    at androidx.fragment.app.FragmentStateManager.moveToExpectedState(FragmentStateManager.java:282)
    at androidx.fragment.app.FragmentManager.executeOpsTogether(FragmentManager.java:2189)
    at androidx.fragment.app.FragmentManager.removeRedundantOperationsAndExecute(FragmentManager.java:2100)
    at androidx.fragment.app.FragmentManager.execPendingActions(FragmentManager.java:2002)
    at androidx.fragment.app.FragmentManager$5.run(FragmentManager.java:524)
    at android.os.Handler.handleCallback(Handler.java:883)
    at android.os.Handler.dispatchMessage(Handler.java:100)
    at android.os.Looper.loop(Looper.java:214)
    at android.app.ActivityThread.main(ActivityThread.java:7357)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:925)
MuntashirAkon commented 3 years ago

At the time opened this issue, ONLY this app wasn't backing up. Now, with the latest version I can't restore ANY app, thus can't conduct any tests.

Logs are saying that you're in the no-root mode.

Cyberavater commented 3 years ago

Which is wrong, I AM in root mode.

MuntashirAkon commented 3 years ago

Which is wrong, I AM in root mode.

Relaunch AM and take logs during that period.

Cyberavater commented 3 years ago

Which is wrong, I AM in root mode.

Relaunch AM and take logs during that period.

Every relaunch of AM gives me a Keystore prompt which I'm not interested in.

E/KS: Could not get encrypted password
java.security.ProviderException: Keystore operation failed
    at android.security.keystore.AndroidKeyStoreKeyGeneratorSpi.engineGenerateKey(AndroidKeyStoreKeyGeneratorSpi.java:332)
    at javax.crypto.KeyGenerator.generateKey(KeyGenerator.java:612)
    at io.github.muntashirakon.AppManager.crypto.ks.CompatUtil.getAesGcmLocalProtectionKey(CompatUtil.java:116)
    at io.github.muntashirakon.AppManager.crypto.ks.CompatUtil.getEncryptedData(CompatUtil.java:215)
    at io.github.muntashirakon.AppManager.crypto.ks.KeyStoreManager.getEncryptedPassword(KeyStoreManager.java:430)
    at io.github.muntashirakon.AppManager.crypto.ks.KeyStoreManager.savePass(KeyStoreManager.java:399)
    at io.github.muntashirakon.AppManager.crypto.ks.KeyStoreManager.generateAndDisplayKeyStorePassword(KeyStoreManager.java:108)
    at io.github.muntashirakon.AppManager.crypto.AuthenticationActivity.onCreate(AuthenticationActivity.java:68)
    at android.app.Activity.performCreate(Activity.java:7824)
    at android.app.Activity.performCreate(Activity.java:7813)
    at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1307)
    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3245)
    at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3409)
    at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:83)
    at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)
    at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)
    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2016)
    at android.os.Handler.dispatchMessage(Handler.java:107)
    at android.os.Looper.loop(Looper.java:214)
    at android.app.ActivityThread.main(ActivityThread.java:7357)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:925)
Caused by: android.security.KeyStoreException: System error
    at android.security.KeyStore.getKeyStoreException(KeyStore.java:1279)
    ... 23 more
E/ROOT
android.os.RemoteException: AMService not running.
    at io.github.muntashirakon.AppManager.ipc.IPCUtils$AMServiceConnectionWrapper.getServiceSafe(IPCUtils.java:185)
    at io.github.muntashirakon.AppManager.ipc.IPCUtils$AMServiceConnectionWrapper.getAmService(IPCUtils.java:171)
    at io.github.muntashirakon.AppManager.ipc.IPCUtils.getAmService(IPCUtils.java:44)
    at io.github.muntashirakon.AppManager.servermanager.LocalServer.launchAmService(LocalServer.java:60)
    at io.github.muntashirakon.AppManager.runner.RunnerUtils.autoDetectRootOrAdb(RunnerUtils.java:166)
    at io.github.muntashirakon.AppManager.runner.RunnerUtils.setModeOfOps(RunnerUtils.java:190)
    at io.github.muntashirakon.AppManager.crypto.AuthenticationActivity.lambda$handleModeOfOps$2$AuthenticationActivity(AuthenticationActivity.java:107)
    at io.github.muntashirakon.AppManager.crypto.-$$Lambda$AuthenticationActivity$RV-B2vNmyKZyexsvVCuRvnoLqh4.run(Unknown Source:2)
    at java.lang.Thread.run(Thread.java:919)
MuntashirAkon commented 3 years ago

Every relaunch of AM gives me a Keystore prompt which I'm not interested in.

Which seems to be the primary issue. It appears that for some reason, Android KeyStore is failing to generate a key. Can you try clearing app data or reinstalling the app. It might be caused due to the presence of debug symbols.

If it doesn't work, take logs again but without any filter and in the logs, search for Cannot connect to keystore.

Cyberavater commented 3 years ago

Uninstalled>Restarted>Installed. Still, nothing is changed. Apart from that, AM can't locate backup volume.

On Mon, Jul 26, 2021, 2:31 PM Muntashir Al-Islam @.***> wrote:

Every relaunch of AM gives me a Keystore prompt which I'm not interested in.

Which seems to be the primary issue. It appears that for some reason, Android KeyStore is failing to generate a key. Can you try clearing app data or reinstalling the app. It might be caused due to the presence of debug symbols.

If it doesn't work, take logs again but without any filter and in the logs, search for Cannot connect to keystore.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MuntashirAkon/AppManager/issues/473#issuecomment-886494276, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHVSMF437ZZRJ4XSPQ5LFNDTZUMMVANCNFSM46H7Q4JQ .

MuntashirAkon commented 3 years ago

Uninstalled>Restarted>Installed. Still, nothing is changed.

Send me some logs during the initialisation. They must be unfiltered.

Apart from that, AM can't locate backup volume.

What do you mean?

Cyberavater commented 3 years ago

Could you try again with the latest debug? There's a slight chance that App Manager will display false-positive results, be sure to check if the files are actually present after the backup is finished.

:56:38.382 D/Runner: RootShellRunner
:56:39.736 D/IPCUtils: service onServiceConnected
:56:39.743 D/IPCUtils: service onResponseReceived
:56:41.070 D/MVM: New instance created
:56:45.064 E/MetadataManager: java.io.FileNotFoundException: Cannot find meta_v2.am.json
:56:46.415 E/MetadataManager: java.io.FileNotFoundException: Cannot find meta_v2.am.json
:56:47.678 E/MetadataManager: java.io.FileNotFoundException: Cannot find meta_v2.am.json
:57:05.068 D/BackupManager: Package: com.mobile.legends, user: 0
:57:05.216 E/OsEnvironment: [StorageVolume: Internal shared storage, StorageVolume: SD card (C97F-1415)]
:57:05.230 E/OsEnvironment: [StorageVolume: Internal shared storage, StorageVolume: SD card (C97F-1415)]
:57:05.241 E/OsEnvironment: [StorageVolume: Internal shared storage, StorageVolume: SD card (C97F-1415)]
:04:40.494 D/IPCUtils: service onServiceDisconnected
:04:40.497 D/IPCUtils: service onResponseReceived
:07:50.294 D/Cache: Deleted 0 images.

Be sure to check if the files are actually present after the backup is finished.

The backup is present, but it's an old one, from when it was working perfectly

Cyberavater commented 3 years ago

Trying again after deleting the file

E/MetadataManager
java.io.FileNotFoundException: Cannot find com.mobile.legends
    at io.github.muntashirakon.io.Path.findFile(Path.java:208)
    at io.github.muntashirakon.AppManager.backup.BackupFiles.getPackagePath(BackupFiles.java:47)
    at io.github.muntashirakon.AppManager.backup.MetadataManager.getBackupFiles(MetadataManager.java:222)
    at io.github.muntashirakon.AppManager.backup.MetadataManager.hasAnyMetadata(MetadataManager.java:183)
    at io.github.muntashirakon.AppManager.backup.BackupDialogFragment.onCreateDialog(BackupDialogFragment.java:127)
    at androidx.fragment.app.DialogFragment.prepareDialog(DialogFragment.java:644)
    at androidx.fragment.app.DialogFragment.onGetLayoutInflater(DialogFragment.java:558)
    at androidx.fragment.app.Fragment.performGetLayoutInflater(Fragment.java:1654)
    at androidx.fragment.app.FragmentStateManager.createView(FragmentStateManager.java:493)
    at androidx.fragment.app.FragmentStateManager.moveToExpectedState(FragmentStateManager.java:282)
    at androidx.fragment.app.FragmentManager.executeOpsTogether(FragmentManager.java:2189)
    at androidx.fragment.app.FragmentManager.removeRedundantOperationsAndExecute(FragmentManager.java:2100)
    at androidx.fragment.app.FragmentManager.execPendingActions(FragmentManager.java:2002)
    at androidx.fragment.app.FragmentManager$5.run(FragmentManager.java:524)
    at android.os.Handler.handleCallback(Handler.java:883)
    at android.os.Handler.dispatchMessage(Handler.java:100)
    at android.os.Looper.loop(Looper.java:214)
    at android.app.ActivityThread.main(ActivityThread.java:7357)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:925)

No files exist afterward.

ainola commented 3 years ago

I am also having this issue. Some apps just refuse to back up. For instance, Element will not back up on both Lineage 18.1 as well as another phone running the stock Sony firmware (android 9).

MuntashirAkon commented 3 years ago

I am also having this issue. Some apps just refuse to back up. For instance, Element will not back up on both Lineage 18.1 as well as another phone running the stock Sony firmware (android 9).

Apps with KeyStore items are disabled by default. You've to enable it in the settings (but then, restoring apps with KeyStore may not work).

ainola commented 3 years ago

Ah, I see. I apologize, this bug seems then to be about something else. Sorry about that!

Cyberavater commented 2 years ago

Some more issues I caught.

  1. Maybe because of the illegal folder name when using multiple backups ':' in the folder name.

    E/BatchOpsManager
io.github.muntashirakon.AppManager.backup.BackupException: Backup failed
at io.github.muntashirakon.AppManager.backup.BackupManager.backup(BackupManager.java:93)
at io.github.muntashirakon.AppManager.batchops.BatchOpsManager.lambda$opBackupRestore$0$BatchOpsManager(BatchOpsManager.java:288)
at io.github.muntashirakon.AppManager.batchops.-$$Lambda$BatchOpsManager$fRbL-cOU1e30grqu_qUYfkugpEc.run(Unknown Source:10)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:462)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.lang.Thread.run(Thread.java:923)
Caused by: java.io.IOException: Could not create directory named 0_11_10_21_6:33_PM
at io.github.muntashirakon.io.Path.findOrCreateDirectory(Path.java:459)
at io.github.muntashirakon.AppManager.backup.BackupFiles.getFreshBackupPath(BackupFiles.java:224)
at io.github.muntashirakon.AppManager.backup.BackupFiles.getFreshBackupPaths(BackupFiles.java:213)
at io.github.muntashirakon.AppManager.backup.BackupManager.backup(BackupManager.java:85)
... 7 more

  2. Seems like something with keys? Older version backed up successfully.

    E/BatchOpsManager
io.github.muntashirakon.AppManager.backup.BackupException: Could not cache .10197_chr_USRPKEY_com+^x8bit+^bitwarden+^xamarinessentials
at io.github.muntashirakon.AppManager.backup.BackupOp.backupKeyStore(BackupOp.java:333)
at io.github.muntashirakon.AppManager.backup.BackupOp.runBackup(BackupOp.java:170)
at io.github.muntashirakon.AppManager.backup.BackupManager.backup(BackupManager.java:89)
at io.github.muntashirakon.AppManager.batchops.BatchOpsManager.lambda$opBackupRestore$0$BatchOpsManager(BatchOpsManager.java:288)
at io.github.muntashirakon.AppManager.batchops.-$$Lambda$BatchOpsManager$fRbL-cOU1e30grqu_qUYfkugpEc.run(Unknown Source:10)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:462)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.lang.Thread.run(Thread.java:923)
Caused by: java.io.FileNotFoundException: Cannot find file:///data/misc/keystore/user_0/.10197_chr_USRPKEY_com+^x8bit+^bitwarden+^xamarinessentials
at io.github.muntashirakon.io.Path.findFile(Path.java:379)
at io.github.muntashirakon.AppManager.backup.BackupOp.backupKeyStore(BackupOp.java:329)
..
MuntashirAkon commented 2 years ago

Maybe because of the illegal folder name when using multiple backups ':' in the folder name.

It seems so.

Seems like something with keys? Older version backed up successfully.

If /data/misc/keystore/user_0/.10197_chr_USRPKEY_com+^x8bit+^bitwarden+^xamarinessentials exist, it might be the case that it is using regular File instead of our ProxyFile to read the file. I will look into this.

MuntashirAkon commented 2 years ago
  1. Maybe because of the illegal folder name when using multiple backups ':' in the folder name.

Fixed in 68600c86

MuntashirAkon commented 2 years ago

2. Seems like something with keys? Older version backed up successfully.

Fixed in 76df9c92

Cyberavater commented 2 years ago

Restore apps with the same domain name in one single go fails. Ex: com.facebook.katana (Facebook) and com.facebook.orca (Messenger) restore will fail, if restoring in one go (batch).

Cyberavater commented 2 years ago

App Name: Mobile Legends AM version: 1322 Backup Volume: ExtSDCard Log:

E/BackupOp: Failed to backup data directory at /storage/emulated/0/Android/data/com.mobile.legends
io.github.muntashirakon.AppManager.backup.BackupException: Failed to backup data directory at /storage/emulated/0/Android/data/com.mobile.legends
  at io.github.muntashirakon.AppManager.backup.BackupOp.backupData([BackupOp.java:258](http://BackupOp.java:258))
  at io.github.muntashirakon.AppManager.backup.BackupOp.runBackup([BackupOp.java:163](http://BackupOp.java:163))
  at io.github.muntashirakon.AppManager.backup.BackupManager.backup([BackupManager.java:90](http://BackupManager.java:90))
  at io.github.muntashirakon.AppManager.batchops.BatchOpsManager.lambda$opBackupRestore$0$BatchOpsManager([BatchOpsManager.java:269](http://BatchOpsManager.java:269))
  at io.github.muntashirakon.AppManager.batchops.-$$Lambda$BatchOpsManager$fRbL-cOU1e30grqu_[qUYfkugpEc.run](http://qUYfkugpEc.run)(Unknown Source:10)
  at java.util.concurrent.Executors$RunnableAdapter.call([Executors.java:462](http://Executors.java:462))
  at [java.util.concurrent.FutureTask.run](http://java.util.concurrent.FutureTask.run)([FutureTask.java:266](http://FutureTask.java:266))
  at java.util.concurrent.ThreadPoolExecutor.runWorker([ThreadPoolExecutor.java:1167](http://ThreadPoolExecutor.java:1167))
  at java.util.concurrent.ThreadPoolExecutor$[Worker.run](http://Worker.run)([ThreadPoolExecutor.java:641](http://ThreadPoolExecutor.java:641))
  at [java.lang.Thread.run](http://java.lang.Thread.run)([Thread.java:919](http://Thread.java:919))
Caused by: java.io.IOException: java.io.FileNotFoundException: /storage/C97F-1415/AppManager/.tmp/backup_1623079911633/data2.tar.gz.0: open failed: EACCES (Permission denied)
  at io.github.muntashirakon.io.SplitOutputStream.checkCurrentStream([SplitOutputStream.java:97](http://SplitOutputStream.java:97))
  at io.github.muntashirakon.io.SplitOutputStream.write([SplitOutputStream.java:65](http://SplitOutputStream.java:65))
  at java.io.BufferedOutputStream.flushBuffer([BufferedOutputStream.java:82](http://BufferedOutputStream.java:82))
  at java.io.BufferedOutputStream.flush([BufferedOutputStream.java:140](http://BufferedOutputStream.java:140))
  at java.io.FilterOutputStream.close([FilterOutputStream.java:158](http://FilterOutputStream.java:158))
  at org.apache.commons.compress.compressors.gzip.GzipCompressorOutputStream.close([GzipCompressorOutputStream.java:196](http://GzipCompressorOutputStream.java:196))
  at io.github.muntashirakon.AppManager.utils.TarUtils.create([TarUtils.java:117](http://TarUtils.java:117))
  at io.github.muntashirakon.AppManager.backup.BackupOp.backupData([BackupOp.java:254](http://BackupOp.java:254))
  ... 9 more
  Suppressed: java.io.IOException: java.io.FileNotFoundException: /storage/C97F-1415/AppManager/.tmp/backup_1623079911633/data2.tar.gz.0: open failed: EACCES (Permission denied)
      at io.github.muntashirakon.io.SplitOutputStream.checkCurrentStream([SplitOutputStream.java:97](http://SplitOutputStream.java:97))
      at io.github.muntashirakon.io.SplitOutputStream.write([SplitOutputStream.java:65](http://SplitOutputStream.java:65))
      at java.io.BufferedOutputStream.flushBuffer([BufferedOutputStream.java:82](http://BufferedOutputStream.java:82))
      at java.io.BufferedOutputStream.flush([BufferedOutputStream.java:140](http://BufferedOutputStream.java:140))
      at java.io.FilterOutputStream.close([FilterOutputStream.java:158](http://FilterOutputStream.java:158))
      at io.github.muntashirakon.AppManager.utils.TarUtils.create([TarUtils.java:78](http://TarUtils.java:78))
      ... 10 more
  Caused by: java.io.FileNotFoundException: /storage/C97F-1415/AppManager/.tmp/backup_1623079911633/data2.tar.gz.0: open failed: EACCES (Permission denied)
      at libcore.io.IoBridge.open([IoBridge.java:496](http://IoBridge.java:496))
      at java.io.FileOutputStream.<init>([FileOutputStream.java:235](http://FileOutputStream.java:235))
      at java.io.FileOutputStream.<init>([FileOutputStream.java:186](http://FileOutputStream.java:186))
      at io.github.muntashirakon.io.ProxyOutputStream.<init>([ProxyOutputStream.java:31](http://ProxyOutputStream.java:31))
      at io.github.muntashirakon.io.SplitOutputStream.checkCurrentStream([SplitOutputStream.java:93](http://SplitOutputStream.java:93))
      ... 15 more
  Caused by: android.system.ErrnoException: open failed: EACCES (Permission denied)
      at libcore.io.Linux.open(Native Method)
      at libcore.io.ForwardingOs.open([ForwardingOs.java:167](http://ForwardingOs.java:167))
      at libcore.io.BlockGuardOs.open([BlockGuardOs.java:252](http://BlockGuardOs.java:252))
      at libcore.io.ForwardingOs.open([ForwardingOs.java:167](http://ForwardingOs.java:167))
      at android.app.ActivityThread$AndroidOs.open([ActivityThread.java:7256](http://ActivityThread.java:7256))
      at libcore.io.IoBridge.open([IoBridge.java:482](http://IoBridge.java:482))
      ... 19 more
Caused by: java.io.FileNotFoundException: /storage/C97F-1415/AppManager/.tmp/backup_1623079911633/data2.tar.gz.0: open failed: EACCES (Permission denied)
  at libcore.io.IoBridge.open([IoBridge.java:496](http://IoBridge.java:496))
  at java.io.FileOutputStream.<init>([FileOutputStream.java:235](http://FileOutputStream.java:235))
  at java.io.FileOutputStream.<init>([FileOutputStream.java:186](http://FileOutputStream.java:186))
  at io.github.muntashirakon.io.ProxyOutputStream.<init>([ProxyOutputStream.java:31](http://ProxyOutputStream.java:31))
  at io.github.muntashirakon.io.SplitOutputStream.checkCurrentStream([SplitOutputStream.java:93](http://SplitOutputStream.java:93))
  ... 16 more
Caused by: android.system.ErrnoException: open failed: EACCES (Permission denied)
  at libcore.io.Linux.open(Native Method)
  at libcore.io.ForwardingOs.open([ForwardingOs.java:167](http://ForwardingOs.java:167))
  at libcore.io.BlockGuardOs.open([BlockGuardOs.java:252](http://BlockGuardOs.java:252))
  at libcore.io.ForwardingOs.open([ForwardingOs.java:167](http://ForwardingOs.java:167))
  at android.app.ActivityThread$AndroidOs.open([ActivityThread.java:7256](http://ActivityThread.java:7256))
  at libcore.io.IoBridge.open([IoBridge.java:482](http://IoBridge.java:482))
  ... 20 more

This app still doesn't backup on latest debug, not sure why it's closed.

Another new observation is that AM fails to restore any other apps after failing to backup ML. AM's restore feature is functional again after restarting AM.

Log:

2021-12-15 01:58:34.027 E/MetadataManager
java.io.FileNotFoundException: Cannot find file:///storage/0808-0813/AppManager/com.mobile.legends
    at io.github.muntashirakon.io.Path.findFile(Path.java:379)
    at io.github.muntashirakon.AppManager.backup.BackupFiles.getPackagePath(BackupFiles.java:47)
    at io.github.muntashirakon.AppManager.backup.MetadataManager.getBackupFiles(MetadataManager.java:221)
    at io.github.muntashirakon.AppManager.backup.MetadataManager.hasAnyMetadata(MetadataManager.java:182)
    at io.github.muntashirakon.AppManager.backup.BackupDialogFragment.onCreateDialog(BackupDialogFragment.java:129)
    at androidx.fragment.app.DialogFragment.prepareDialog(DialogFragment.java:644)
    at androidx.fragment.app.DialogFragment.onGetLayoutInflater(DialogFragment.java:558)
    at androidx.fragment.app.Fragment.performGetLayoutInflater(Fragment.java:1654)
    at androidx.fragment.app.FragmentStateManager.createView(FragmentStateManager.java:493)
    at androidx.fragment.app.FragmentStateManager.moveToExpectedState(FragmentStateManager.java:282)
    at androidx.fragment.app.FragmentManager.executeOpsTogether(FragmentManager.java:2189)
    at androidx.fragment.app.FragmentManager.removeRedundantOperationsAndExecute(FragmentManager.java:2100)
    at androidx.fragment.app.FragmentManager.execPendingActions(FragmentManager.java:2002)
    at androidx.fragment.app.FragmentManager$5.run(FragmentManager.java:524)
    at android.os.Handler.handleCallback(Handler.java:938)
    at android.os.Handler.dispatchMessage(Handler.java:99)
    at android.os.Looper.loop(Looper.java:223)
    at android.app.ActivityThread.main(ActivityThread.java:7666)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)
2021-12-15 01:58:36.505 D/BackupManager: Package: com.mobile.legends, user: 0
2021-12-15 01:58:36.628 E/OsEnvironment: [StorageVolume: Internal shared storage, StorageVolume: SD card (0808-0813)]
2021-12-15 01:58:36.652 E/OsEnvironment: [StorageVolume: Internal shared storage, StorageVolume: SD card (0808-0813)]
2021-12-15 01:58:36.660 E/OsEnvironment: [StorageVolume: Internal shared storage, StorageVolume: SD card (0808-0813)]
2021-12-15 02:32:25.145 D/IPCUtils: service onServiceDisconnected
2021-12-15 02:32:25.165 D/IPCUtils: service onResponseReceived
2021-12-15 02:32:26.960 E/BatchOpsManager
io.github.muntashirakon.AppManager.backup.BackupException: Failed to backup data directory at /storage/emulated/0/Android/data/com.mobile.legends
    at io.github.muntashirakon.AppManager.backup.BackupOp.backupData(BackupOp.java:295)
    at io.github.muntashirakon.AppManager.backup.BackupOp.runBackup(BackupOp.java:168)
    at io.github.muntashirakon.AppManager.backup.BackupManager.backup(BackupManager.java:89)
    at io.github.muntashirakon.AppManager.batchops.BatchOpsManager.lambda$opBackupRestore$0$BatchOpsManager(BatchOpsManager.java:288)
    at io.github.muntashirakon.AppManager.batchops.-$$Lambda$BatchOpsManager$fRbL-cOU1e30grqu_qUYfkugpEc.run(Unknown Source:10)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:462)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
    at java.lang.Thread.run(Thread.java:923)
Caused by: java.io.FileNotFoundException: The file cannot be opened for reading.
    at io.github.muntashirakon.io.ProxyInputStream.<init>(ProxyInputStream.java:31)
    at io.github.muntashirakon.io.Path.openInputStream(Path.java:812)
    at io.github.muntashirakon.AppManager.utils.TarUtils.create(TarUtils.java:106)
    at io.github.muntashirakon.AppManager.backup.BackupOp.backupData(BackupOp.java:290)
    ... 9 more
    Suppressed: java.io.IOException: This archive contains unclosed entries.
        at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.finish(TarArchiveOutputStream.java:276)
        at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.close(TarArchiveOutputStream.java:294)
        at io.github.muntashirakon.AppManager.utils.TarUtils.create(TarUtils.java:85)
        ... 10 more
2021-12-15 02:33:04.254 D/BackupManager: Package: com.rhmsoft.edit.pro, user: 0
2021-12-15 02:33:04.329 D/RestoreOp: Close called
2021-12-15 02:33:04.331 E/BatchOpsManager
io.github.muntashirakon.AppManager.backup.BackupException: Could not create staging files
    at io.github.muntashirakon.AppManager.backup.RestoreOp.restoreApkFiles(RestoreOp.java:278)
    at io.github.muntashirakon.AppManager.backup.RestoreOp.runRestore(RestoreOp.java:182)
    at io.github.muntashirakon.AppManager.backup.BackupManager.restore(BackupManager.java:163)
    at io.github.muntashirakon.AppManager.batchops.BatchOpsManager.lambda$opBackupRestore$0$BatchOpsManager(BatchOpsManager.java:294)
    at io.github.muntashirakon.AppManager.batchops.-$$Lambda$BatchOpsManager$fRbL-cOU1e30grqu_qUYfkugpEc.run(Unknown Source:10)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:462)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
    at java.lang.Thread.run(Thread.java:923)
Caused by: java.io.IOException: Could not create file:///data/local/tmp/base.apk with type application/x-invalid-mime-type
    at io.github.muntashirakon.io.Path.createFileAsDirectChild(Path.java:893)
    at io.github.muntashirakon.io.Path.createNewFile(Path.java:251)
    at io.github.muntashirakon.AppManager.backup.RestoreOp.restoreApkFiles(RestoreOp.java:270)
    ... 9 more
2021-12-15 02:33:18.631 D/BackupManager: Package: com.rhmsoft.edit.pro, user: 0
2021-12-15 02:33:18.683 D/RestoreOp: Close called
2021-12-15 02:33:18.684 E/BatchOpsManager
io.github.muntashirakon.AppManager.backup.BackupException: Could not create staging files
    at io.github.muntashirakon.AppManager.backup.RestoreOp.restoreApkFiles(RestoreOp.java:278)
    at io.github.muntashirakon.AppManager.backup.RestoreOp.runRestore(RestoreOp.java:182)
    at io.github.muntashirakon.AppManager.backup.BackupManager.restore(BackupManager.java:163)
    at io.github.muntashirakon.AppManager.batchops.BatchOpsManager.lambda$opBackupRestore$0$BatchOpsManager(BatchOpsManager.java:294)
    at io.github.muntashirakon.AppManager.batchops.-$$Lambda$BatchOpsManager$fRbL-cOU1e30grqu_qUYfkugpEc.run(Unknown Source:10)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:462)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
    at java.lang.Thread.run(Thread.java:923)
Caused by: java.io.IOException: Could not create file:///data/local/tmp/base.apk with type application/x-invalid-mime-type
    at io.github.muntashirakon.io.Path.createFileAsDirectChild(Path.java:893)
    at io.github.muntashirakon.io.Path.createNewFile(Path.java:251)
    at io.github.muntashirakon.AppManager.backup.RestoreOp.restoreApkFiles(RestoreOp.java:270)
    ... 9 more
MuntashirAkon commented 2 years ago

This app still doesn't backup on latest debug, not sure why it's closed.

It's because you specify multiple issues in a single issue. As a result, it becomes very difficult to keep track of the actual issues. You should specify only one issue per post so that the comments could be linear.

MuntashirAkon commented 2 years ago

By the way, how large is the app? It is best that I reproduce the issue myself in order to understand the situation.

Cyberavater commented 2 years ago

hdjdb

Cyberavater commented 2 years ago

From the SwiftBackup log, it seems SB is using 7zip to archive and extract, maybe AM should use this too? (SB operations seem faster to me, please verify using 7zip if AM's performance can be improved) (Official 7zip for Linux was released recently).

MuntashirAkon commented 2 years ago

From the SwiftBackup log, it seems SB is using 7zip to archive and extract, maybe AM should use this too?

Not possible for the same reason why Toybox was discarded. AM has its own file system which is a merger of Linux FS, SAF and some other virtual FS used internally. This is because AM needs a stable file system API unlike SB.

MuntashirAkon commented 2 years ago

Could you retake the logs from the latest debug?

Cyberavater commented 2 years ago 
E/BatchOpsManager
io.github.muntashirakon.AppManager.backup.BackupException: Failed to backup data directory at /storage/emulated/0/Android/data/com.mobile.legends
    at io.github.muntashirakon.AppManager.backup.BackupOp.backupData(BackupOp.java:281)
    at io.github.muntashirakon.AppManager.backup.BackupOp.runBackup(BackupOp.java:168)
    at io.github.muntashirakon.AppManager.backup.BackupManager.backup(BackupManager.java:89)
    at io.github.muntashirakon.AppManager.batchops.BatchOpsManager.lambda$opBackupRestore$0$BatchOpsManager(BatchOpsManager.java:288)
    at io.github.muntashirakon.AppManager.batchops.-$$Lambda$BatchOpsManager$fRbL-cOU1e30grqu_qUYfkugpEc.run(Unknown Source:10)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:462)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
    at java.lang.Thread.run(Thread.java:923)
Caused by: java.io.FileNotFoundException: The file cannot be opened for reading.
    at io.github.muntashirakon.io.ProxyInputStream.<init>(ProxyInputStream.java:31)
    at io.github.muntashirakon.io.Path.openInputStream(Path.java:812)
    at io.github.muntashirakon.AppManager.utils.TarUtils.create(TarUtils.java:106)
    at io.github.muntashirakon.AppManager.backup.BackupOp.backupData(BackupOp.java:276)
    ... 9 more
    Suppressed: java.io.IOException: This archive contains unclosed entries.
        at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.finish(TarArchiveOutputStream.java:276)
        at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.close(TarArchiveOutputStream.java:294)
        at io.github.muntashirakon.AppManager.utils.TarUtils.create(TarUtils.java:85)
        ... 10 more
Cyberavater commented 2 years ago

Could you retake the logs from the latest debug?

Just a friendly reminder.

MuntashirAkon commented 2 years ago

Try again with the latest debug.

Cyberavater commented 2 years ago

Now it backup only 175 MB of 6GB Data.

MuntashirAkon commented 2 years ago

Without errors?

Cyberavater commented 2 years ago

Yes

On Wed, May 11, 2022, 9:04 PM Muntashir Al-Islam @.***> wrote:

Without errors?

— Reply to this email directly, view it on GitHub https://github.com/MuntashirAkon/AppManager/issues/473#issuecomment-1123900774, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHVSMFZAYUBBKXLEJS7FI23VJPD7PANCNFSM46H7Q4JQ . You are receiving this because you authored the thread.Message ID: @.***>

MuntashirAkon commented 2 years ago

Yes

Hmmm, this is even worse, then.

MuntashirAkon commented 2 years ago

Could you send some logs taken while the app was being backed up?

Cyberavater commented 2 years ago 
2022-05-26 17:29:13.581 D/BackupManager: Package: com.mobile.legends, user: 0
2022-05-26 17:29:13.758 E/OsEnvironment: [StorageVolume: Internal shared storage, StorageVolume: Ventoy (4E21-0000), StorageVolume: VTOYEFI (DE2E-30BF)]
2022-05-26 17:29:13.786 E/OsEnvironment: [StorageVolume: Internal shared storage, StorageVolume: Ventoy (4E21-0000), StorageVolume: VTOYEFI (DE2E-30BF)]
2022-05-26 17:29:13.792 E/OsEnvironment: [StorageVolume: Internal shared storage, StorageVolume: Ventoy (4E21-0000), StorageVolume: VTOYEFI (DE2E-30BF)]
2022-05-26 17:29:17.328 E/ComponentBlocker: Could not retrieve info for package com.rt.upleftout with flags 0x2200
Cyberavater commented 2 years ago

Yes

Hmmm, this is even worse, then.

It has nothing to do with app being large it seems. It costed me Lossing important data of other apps to find this out. AM just doesn't backup external data folder.

MuntashirAkon commented 2 years ago

It has nothing to do with app being large it seems. It costed me Lossing important data of other apps to find this out. AM just doesn't backup external data folder.

Thanks for testing. This definitely needs a fix before the final release.

MuntashirAkon commented 2 years ago

It turns out that this issue was introduced in v3.0.0-rc01 via 7b1e30e5b1e7bc1d72014a98d8e26285c8773341. It is now fixed via e06b8a12e64874abbde6240e42d702a69331ad45.

Please check now.

MuntashirAkon commented 2 years ago

The original issue, however, might still exist.

MuntashirAkon commented 2 years ago

Also, if you have earlier backups, they were backed up correctly but App Manager weren't able to restore them properly.