[Bug] (Multi-User) Cannot backup/restore External Data

[BurhanDanger]

Description When in multi user setup, backing up or restoring External Data ( /Android/data) results in an error.

Steps To Reproduce

1. Go to a different user than use 0
2. Select an app ( eg. Aurora Store)
3. Select backup and select External data
4. Shows message:

   
   Error Occurred:
   a: Could not list contents of /storage/emulated/12/Android/data/com.aurora.store

**Expected behavior**
Backup and restore without any error.

**System Information(please complete the following information):**

- Device: Redmi note 9 Pro
- Android Version: A11
- ROM: Fork_LineageOS  
- App's Version: 7.0.0

[BurhanDanger]

Similar issue: https://github.com/machiav3lli/oandbackupx/issues/388

[xxxcrow]

Having the same issue 👋 I'm pretty sure it worked before

About #388 never worked for me. I'm using shelter and their "Main" and "Shelter" file shuttle But i'm pretty sure this is a problem of the file shuttle

[Iey4iej3]

Same here. I use the OAndBackupX in the shelter, and seemingly no file shuttle is apparently necessary for this?

[brainchild0]

I am also affected, unable to complete a simple system backup. I just installed the latest version published to F-Droid, 7.0.0, into a new installation of LineageOS, based on Android 11.

[terxw]

I am also affected, version 7.0

error is

could not list contents of /data/user/...

[Reporter1]

Hi I'm just adding, Im seeing this issues to. Backed up around 200 hundred apps, then remaining 37 (there is 37 notifications in the notifications drop down bar) I have could not list contents of /data/user/0

Latest version from fdroid. Lineage 18.1 Pixel 5

[Iey4iej3]

https://github.com/NeoApplications/Neo-Backup/issues/278 similar error but single user.

[mimir-d]

in case this helps anyone, I made a script for myself to restore backups to a different user. In short, what it does is:

1. install the apk (possibly split)
2. unpack private app data (ie. /data/.../$appid)
3. unpack external data/obb/media (ie. /storage/emulated/.../Android/{data,obb,media})
4. resets selinux contexts, permissions, etc

DISCLAIMER: should you choose to run this script, I am not in any way, shape or form, responsible for any breakage (or possible bricking) of your phone. This is meant more for developer instructions and/or people familiar with android OS.

example run

• params are /path/to/jq and /path/to/$backup_dir/$app_fqdn
• download an arch specific jq like the arm64 one here and place everything in $TMPDIR for example
• note that I only ran this on a pixel 7, android 14, adb root, non-owner user; no other scenarios were tested

panther:/data/local/tmp # ./restore.sh ./jq-linux-arm64 /mnt/user/10/emulated/10/backups/neo_backups/com.quizup.core
Current user id: 10
Props file path: /mnt/user/10/emulated/10/backups/neo_backups/com.quizup.core/2023-11-27-23-00-22-066-user_0.properties
Using backup: /mnt/user/10/emulated/10/backups/neo_backups/com.quizup.core/2023-11-27-23-00-22-066-user_0
--------- Installing APK -------------
Name: com.quizup.core
Total apk size: 59704642
Success: streamed 51154154 bytes
Success: streamed 1454519 bytes
Success: streamed 49498 bytes
Success: streamed 7046471 bytes
Success
---------- Unpacking data ------------
Name: com.quizup.core
Source file: /mnt/user/10/emulated/10/backups/neo_backups/com.quizup.core/2023-11-27-23-00-22-066-user_0/data.tar.gz
Target dir: /data/user/10/com.quizup.core
reowning /data/user/10/com.quizup.core
user remap: u0_a173 -> 1010155
group remap: u0_a173 -> 1010155
>> [...files...]
---------- Unpacking external data ------------
Name: com.quizup.core
Source file: /mnt/user/10/emulated/10/backups/neo_backups/com.quizup.core/2023-11-27-23-00-22-066-user_0/external_files.tar.gz
Target dir: /mnt/user/10/emulated/10/Android/data/com.quizup.core
reowning /mnt/user/10/emulated/10/Android/data/com.quizup.core
user remap: u0_a173 -> 1010155
group remap: u0_a173 -> 1010155
>> [...files...]
---------- Unpacking obb ------------
Name: com.quizup.core
Skipping, no source: /mnt/user/10/emulated/10/backups/neo_backups/com.quizup.core/2023-11-27-23-00-22-066-user_0/obb_files.tar.gz
---------- Unpacking media ------------
Name: com.quizup.core
Skipping, no source: /mnt/user/10/emulated/10/backups/neo_backups/com.quizup.core/2023-11-27-23-00-22-066-user_0/media_files.tar.gz

the script source

• note that it will pick the latest modified date backup, ideally you'd only have 1 subfolder for the dated backup
• a couple of safety checks are done, but otherwise this really should not be run without understanding what it does

#!/system/bin/sh
set -eu
# set -x

if [ "$(id -u)" -ne 0 ]; then
        echo "need root (do not run unless you understand what youre doing)"
        exit 1
fi

if [ $# -ne 2 ]; then
        echo "usage: $0 <path/to/jq> <path/to/backup/folder>"
        exit 1
fi

JQ="$(readlink -f "$1")"
BASE_DIR="$2"

WORKDIR="$(mktemp -d)"
trap "rm -rf $WORKDIR; exit" EXIT SIGINT SIGTERM
cd "$WORKDIR"

USER="$(am get-current-user)"
APP_NAME="$(basename "$BASE_DIR")"
PROPS="$(ls -t "$BASE_DIR"/*.properties | head -1)"
LAST_BACKUP="$(basename "$PROPS" .properties)"
BACKUP_DIR="$BASE_DIR/$LAST_BACKUP"

echo "Current user id: $USER"
echo "Props file path: $PROPS"
echo "Using backup: $BACKUP_DIR"

BASE_APK="$(cat "$PROPS" | $JQ -r ".sourceDir" | xargs basename)"
cp "$BACKUP_DIR/$BASE_APK" .  # copy to workdir, pm doesnt have perms on storage due to selinux

size_parts="$(stat -c %s "$BACKUP_DIR/$BASE_APK")"
split_apks=
for split_entry in $(cat "$PROPS" | $JQ -r ".splitSourceDirs[]"); do
        split_apk_name="$(echo $split_entry | xargs basename)"
        size_parts="$size_parts + $(stat -c %s $BACKUP_DIR/$split_apk_name)"
        split_apks="$split_apks $split_apk_name"
        cp "$BACKUP_DIR/$split_apk_name" .  # copy to workdir, selinux
done
APK_SIZE="$(echo $size_parts | bc -l)"

echo --------- Installing APK -------------
echo "Name: $APP_NAME"
echo "Total apk size: $APK_SIZE"

session=$(pm install-create -S $APK_SIZE --user $USER | grep -oE "[0-9]+")
pm install-write -S "$(stat -c %s $BASE_APK)" $session 0 $BASE_APK
index=1
for split_apk in $split_apks; do
        pm install-write -S "$(stat -c %s $split_apk)" $session $index "$split_apk"
        index="$(echo $index+1 | bc -l)"
done
pm install-commit $session

APP_UID="$(pm list packages -U | grep "$APP_NAME" | grep -oE "uid:[0-9]+" | cut -f2 -d:)"
APP_GID="$APP_UID"

DATA_FILE="$BACKUP_DIR/data.tar.gz"  # assumption
if ! [ -f "$DATA_FILE" ]; then
        echo "backup doesnt have a data.tar.gz; cannot continue, need this for uid/gid guessing"
        exit 1
fi

# divinate the original uid/gid and make a map
read -r ORIG_USER ORIG_GROUP <<<"$(tar -tvf "$DATA_FILE" --list | cut -f2 -d" " | tr / ' ')"
reown() {
        # toybox tar doesnt know --owner-map/--group-map
        echo "reowning $1"
        echo "user remap: $ORIG_USER -> $APP_UID"
        echo "group remap: $ORIG_GROUP -> $APP_GID"

        # need to go thru each (instead of chown -R) because some may in fact have correct user/group
        # (eg. things that are already there and not in the tar.gz backup)
        find "$1" | while read -r name; do
                echo ">> $name"
                if [ "$(stat -c %U "$name")" = "$ORIG_USER" ]; then
                        chown $APP_UID "$name" || true
                fi
                if [ "$(stat -c %G "$name")" = "$ORIG_GROUP" ]; then
                        chgrp $APP_GID "$name" || true
                fi
        done
}

extract_maybe() {
        source="$2"
        target="$3"

        echo ---------- Unpacking $1 ------------
        echo "Name: $APP_NAME"
        if [ -f "$source" ]; then
                echo "Source file: $source"
                echo "Target dir: $target"

                mkdir -p "$target"
                tar xzf "$source" -C "$target" --selinux
                reown "$target"
        else
                echo "Skipping, no source: $source"
        fi
}

extract_maybe "data" "$BACKUP_DIR/data.tar.gz" "/data/user/$USER/$APP_NAME"
extract_maybe "external data" "$BACKUP_DIR/external_files.tar.gz" "/mnt/user/$USER/emulated/$USER/Android/data/$APP_NAME"
extract_maybe "obb" "$BACKUP_DIR/obb_files.tar.gz" "/mnt/user/$USER/emulated/$USER/Android/obb/$APP_NAME"
extract_maybe "media" "$BACKUP_DIR/media_files.tar.gz" "/mnt/user/$USER/emulated/$USER/Android/media/$APP_NAME"

This also avoids the problem in #587 since it installs the apk directly (for me the apk restore in the UI failed like the issue mentioned).