[Bug] Backups not being recognized

[wiltRainbow]

Description Backups can neither be seen on the restore tab nor on the app's individual view. Maybe device specific?

Steps To Reproduce

1. Make a backup of any app
2. See how everything seems to be ok in the backup directory
3. Backups aren't shown by the app

Expected behavior Seeing backups everywhere it is supposed to be shown

Screenshots

https://user-images.githubusercontent.com/109252692/184448550-66b32792-0c86-435f-aa9e-9bf64fe96e98.mp4

some logs

``` [08-12 23:27:05.855 22699:22699 D/k@9:HomeFragment] refreshing [08-12 23:27:05.855 22699:22699 D/k@9:BatchFragment] refreshing [08-12 23:27:05.855 22699:22699 D/k@9:HomeFragment] refreshing [08-12 23:27:05.856 22699:22699 D/k@9:BatchFragment] refreshing [08-12 23:27:05.970 22699:23335 W/I@2:f] refreshbackupList: io.anuke.mindustry [08-12 23:27:05.998 22699:23335 E/c@7:g$a] unexpected: { content://com.android.externalstorage.documents/tree/primary%3ANeoBackup/document/primary%3ANeoBackup%2Fio.anuke.mindustry%2F2022-08-12-22-57-17-666-user_0.properties } [08-12 23:27:05.998 22699:23335 D/b@2:c$c] json: [08-12 23:27:05.999 22699:23335 E/c@7:g$a] unexpected: { file: /tree/primary:NeoBackup/document/primary:NeoBackup/io.anuke.mindustry/2022-08-12-22-57-17-666-user_0.properties = [08-12 23:27:05.999 22699:23335 W/I@7:f] Incomplete backup or wrong structure found in /tree/primary:NeoBackup/document/primary:NeoBackup/io.anuke.mindustry/2022-08-12-22-57-17-666-user_0.properties [08-12 23:27:06.004 22699:23335 W/I@2:f] refreshbackupList: LOGS (1) [08-12 23:27:06.008 22699:23335 I/@27:f] LOGS (1) is not installed [08-12 23:27:06.008 22699:23335 E/j@18:j0] Could not process backup folder for uninstalled application in LOGS (1): java.lang.AssertionError: Backup History is empty and package is not installed. The package is completely unknown? ```

System Information(please complete the following information):

• Device:
• Android Version: 12
• ROM: One UI 4.1
• App's Version: 8.1.1

[wiltRainbow]

{"backupVersionCode":8001,"packageName":"io.anuke.mindustry","packageLabel":"Mindustry","versionName":"5-fdroid-104","versionCode":104,"sourceDir":"/data/app/~~eg0bDuev3kNCxjbOi7dJXg==/io.anuke.mindustry-YnwcGNx5ph7OqjBz-CXTCw==/base.apk","splitSourceDirs":[],"backupDate":"2022-08-12T22:57:17.666","hasApk":true,"hasAppData":true,"iv":[-21,-128,-119,21,63,97,-37,-76,41,87,25,-67,-47,-53,71,94],"cpuArch":"arm64-v8a","permissions":[""],"size":39373924}

This is what the properties file look like

[hg42]

these lines say that the file is empty:

[08-12 23:27:05.998 22699:23335 D/b@2:c$c] json:

[08-12 23:27:05.999 22699:23335 E/c@7:g$a] unexpected: { file: /tree/primary:NeoBackup/document/primary:NeoBackup/io.anuke.mindustry/2022-08-12-22-57-17-666-user_0.properties =

[08-12 23:27:05.999 22699:23335 W/I@7:f]

however, you can read it, so it's not empty.

This probably means, NB cannot read it.

I guess, it's because of the 666 in the file name :-)

If you refresh the package list manually, does it still not show the backup? please log this, too.

[wiltRainbow]

Those are logs from manually refreshing 😕 Tested again and nothing changed

full log

[machiav3lli]

It seems like the storage content is somehow "shadowed". Do you have any magisk modules or "special" apps? Please provide a list of such issues

[hg42]

you could also try enabling

shadowRootFile and allowShadowingDefault (to allow the other one)

to try another access method (via root instead 8f SAF, not guarantied, falls back to SAF).

Or, may be you already enabled both, then disable shadowRootFile, to use SAF instead.

[hg42]

Those are logs from manually refreshing 😕

thanks...

so it's a permanent state.

Last time I had such a kind of problem (able to write, but not to read?), I searched for a reason for some time and at the end it was magically solved by rebooting the phone. I still don't know what the real reason was. I guess some caching hiccup. So just to be sure, please reboot :-)

[hg42]

the output of the following commands in a root shell could shed some light on the permissions:

ls -dlZ /sdcard/NeoBackup/io.anuke.mindustry
ls -AlZ /sdcard/NeoBackup/io.anuke.mindustry
ls -dlZ /storage/self/primary/NeoBackup/io.anuke.mindustry
ls -AlZ /storage/self/primary/NeoBackup/io.anuke.mindustry

[wiltRainbow]

It seems like the storage content is somehow "shadowed". Do you have any magisk modules or "special" apps? Please provide a list of such issues

The only Magisk module I use is Advanced Charging Controller. But this is a Custom ROM (which I didn't create), so it's possible that there is something not working properly.

So just to be sure, please reboot :-)

Unluckily didn't make a change, but enabling shadowRootFile and allowShadowingDefault made the backups appear :D

[wiltRainbow]

I didn't know this functionality exists, thank you very much! Should this issue be closed now?

[wiltRainbow]

the output of the following commands in a root shell could shed some light on the permissions:

ls -dlZ /sdcard/NeoBackup/io.anuke.mindustry
ls -AlZ /sdcard/NeoBackup/io.anuke.mindustry
ls -dlZ /storage/self/primary/NeoBackup/io.anuke.mindustry
ls -AlZ /storage/self/primary/NeoBackup/io.anuke.mindustry

~ $ ls -dlZ /sdcard/NeoBackup/io.anuke.mindustry
drwxrwx--- 3 root everybody ? 3488 Aug 13 09:37 /sdcard/NeoBackup/io.anuke.mindustry
~ $ ls -AlZ /sdcard/NeoBackup/io.anuke.mindustry
total 4
drwxrwx--- 2 root everybody ? 3488 Aug 12 22:57 2022-08-12-22-57-17-666-user_0
-rw-rw---- 1 root everybody ?  453 Aug 12 22:57 2022-08-12-22-57-17-666-user_0.properties
~ $ ls -dlZ /storage/self/primary/NeoBackup/io.anuke.mindustry
drwxrwx--- 3 root everybody ? 3488 Aug 13 09:37 /storage/self/primary/NeoBackup/io.anuke.mindustry
~ $ ls -AlZ /storage/self/primary/NeoBackup/io.anuke.mindustry
total 4
drwxrwx--- 2 root everybody ? 3488 Aug 12 22:57 2022-08-12-22-57-17-666-user_0
-rw-rw---- 1 root everybody ?  453 Aug 12 22:57 2022-08-12-22-57-17-666-user_0.properties

[hg42]

Should this issue be closed now?

not really, because it's a workaround, but the real problem is still unknown. If you would like to help finding it, this would be nice. Though, currently, I have no clue what to do/test next.

The option shadowRootFile searches for the cryptic SAF url of the backup directory (or any other file tree), in your case:

/tree/primary:NeoBackup

by taking the last part NeoBackup and searching it in some well-known directories. If it finds it, all files inside are accessed by root commands, otherwise it falls back to using SAF.

Unfortunately there are some ROMs where shadowRootFile hangs, so it's not enabled by default. Also, after some of the latest changes, shadowRootFile is no more significantly faster than SAF (sometimes even slower).

Though, SAF has it's own hiccups, e.g. if people/ROMs/whatever remove the "Files" app (a.k.a. DocumentsUI). Lately MixPlorer Magisk module did that.

[hg42]

thanks...

drwxrwx--- 3 root everybody ? 3488 Aug 13 09:37 /sdcard/NeoBackup/io.anuke.mindustry
drwxrwx--- 2 root everybody ? 3488 Aug 12 22:57 2022-08-12-22-57-17-666-user_0
-rw-rw---- 1 root everybody ?  453 Aug 12 22:57 2022-08-12-22-57-17-666-user_0.properties

drwxrwx--- 3 root everybody ? 3488 Aug 13 09:37 /storage/self/primary/NeoBackup/io.anuke.mindustry
drwxrwx--- 2 root everybody ? 3488 Aug 12 22:57 2022-08-12-22-57-17-666-user_0
-rw-rw---- 1 root everybody ?  453 Aug 12 22:57 2022-08-12-22-57-17-666-user_0.properties

1. the lines for /sdcard/ and /storage/self/primary/ have the same attributes, so there is no difference in accessing them (it was thought to compare your manual access vs. NB)
2. the rw flags are set for all files, so they should be readable and writable
3. the selinux part is ? which means we cannot read the selinux contexts from the files. The question mark is for a case, that shouldn't exist (I wrote something like "when does this happen?"). May be that's a problem...but not sure. Basically those files are created by NB and therefore should be readable, too. Or in other words whatever selinux context (or other attributes) might be set when writing those files, should also be ok for reading them.

[mddvul22]

I seem to be having the same problem, but I'm not positive as my issue may be slightly different. I had a bunch of Neo Backup backups on my phone that was running Lineage 18. Then, last night I rebuilt my phone using Lineage 19. Oddly, Neo Backup sees some of the backups in the backup directory, but others, it does not. For example, it doesn't see that I have a Fennec (Firefox) backup in there. It doesn't see that I have a backup for "Password Store", "SMS Backup+", "T-UI", "Battery Charge Limit", or "OpenKeyChain", either. There may be a few others. Oddly, it does see some backups, like K-9 Mail, for example. Not sure what the logic is behind whether it sees a backup in that directory, or not.

[hg42]

the key would be to see what they have in common. This might be based on permissions/attributes, or may be an indicator could be the backup date. Or may be the invisible backups are all incomplete in some way.

Do you have a work profile? using shelter or similar?

if you keep more than single backups, are tgere apps where an old backup is visible and axa newer isn't? or vice versa?

the output of

ls -RAlZ /the/top/backup/directory/path

might help (if you can disclose app names)

otherwise doing that for a few of the visible packages and a few of the invisible packages may be enough.

[hg42]

you could also try enabling

shadowRootFile and allowShadowingDefault (to allow the other one)

to try another access method (via root instead 8f SAF, not guarantied, falls back to SAF).

Or, may be you already enabled both, then disable shadowRootFile, to use SAF instead.

and this doesn't work? or disabling them if you have both enabled

[mddvul22]

Upon further inspection, my problem appears to be this one: https://github.com/NeoApplications/Neo-Backup/issues/561

All of the missing backups are older backups. The time stamps on the folders of these backups show they were made back in 2021.

[hg42]

ok, I once added some compatibility code to be able to restore v7 backups, but it seems like this code fails for your backups.

please attach a few of the failing *.properties files. May be I can find the reason.

It's also possible that some newer changes to the file format (adding size etc.?) need some more compatibility tweaks.

[hg42]

if you have access to

8.0.3-beta1

https://t.me/neo_backup/36983

you could try it to see, if it can see those old backups.

[telmob]

if you have access to

8.0.3-beta1

https://t.me/neo_backup/36983

you could try it to see, if it can see those old backups.

Still not working for me, sadly. Even tried to copy the backup folder to another location, but without success. i had to install an older version for it to work (v 8.1.0)