[idevicebackup2] Restoring an incremental, encrypted backup with iOS 14.3 -> 15.1 fails to reinstall applications

[intelfx]

I have two devices: iPhone X (iPhone10,6, A1901) and iPad Pro 4th (iPad8,12, A2232) which I'm trying to update from iOS/iPadOS 14.3 to 15.1.

I made backups of both devices with idevicebackup2 backup (incremental backups into existing backup folders, with encryption on). After an OTA upgrade followed by a full reset and a restore with idevicebackup2 restore --interactive --system --settings, previously installed applications failed to reinstall.

[mexmer]

applications needs to be downloaded from appstore, they are not part of backup, only placeholder icons, and application list and data are inside backup. to restore application, your phone needs to have internet connection, and already setup appleid, including appstore

[intelfx]

Yes, I am aware how this works, but it just does not happen at all, no placeholder icons and nothing happens after the restore is completed.

Additionally, after I attempted to reinstall the applications by hand, it appeared that their data and settings were not saved either.

[mexmer]

if they were not saved in backup, it cannot possibly restore it then. tbh. i never use incremental backup, i always do full backup, so i never run into this issue.

you can check your manifest.plist and info.plist inside backup directory, if there is application dictionary, and if application entries contain purchase info (there should be account to which is application bound)

[nikias]

The problem is not because of incremental backups. The backups system will always have a full backup state if you do an incremental backup on top of a previous full backup (that might have been obtained with another incremental backup before hand). If the backup system finds inconsistencies it will do a full backup automatically. Applications themselves are not part of the backup, that is true. However there is application information in the contained Info.plist, and that information is used to write a RestoreApplications.plist file that is written to the device upon restore, and that should, after the restore is finished and the device rebooted, trigger app installation. I think it's true though that the device needs to be provisioned with an Apple ID, otherwise it will not be able to reinstall applications. Manual install should actually result in restore of the corresponding app data, given they were part of the backup. If you still have the backup data, you can open the Manifest.db with sqlite3 command line tool and do a query for a known app, e.g.

select * from Files where domain like 'AppDomain-BUNDLEID';

where you obviously replace BUNDLEID with the actual app bundle identifier. You can also query all app-related files using

select * from Files where domain like 'AppDomain-%';

to see what is included in the backup.

Hope that helps.

[mexmer]

i would still check, if there is iTunesMetada entry under application key inside info.plist, if there is none, or is empty, it will restore nothing

application entry looks like

bundleid

<dict>
  <key>ApplicationSINF</key>
  <data>base64 sinfdata</data>
  <key>PlaceholderIcon</key>
  <data>base64 png icon</data>
  <key>iTunesMetadata</key>
  <data>base64 itunes purchase data</data>
</dict>

iTunesMetadata is also plist, which contains downloadinfo and accountinfo and other stuff related to appstore. if it's not present, app will not be restored.

[intelfx]

@mexmer Info.plist seems to be intact (i. e. it does contain the keys for all applications I care about, including iTunesMetadata sections). I did not look inside the base64'd blobs in the iTunesMetadata sections.

@nikias Manifest.db doesn't look like an sqlite3 database — I saw something about Apple starting to encrypt the manifest starting with some iOS version, I guess that must be it?

$ pwd
/mnt/data/Backups/Mobile/shiny/e26241378c6284560f192dfe1d477485ee63b43f

$ sqlite3 Manifest.db
SQLite version 3.38.0 2022-02-22 18:58:40
Enter ".help" for usage hints.
sqlite> select * from Files where domain like 'AppDomain-%';
Parse error: file is not a database (26)
sqlite> 

$ file Manifest.db
Manifest.db: data

[nikias]

@intelfx oh right, I forgot about that... my bad. But in any case the Info.plist seems to be including the required app data, so we might want to figure out if it fails to write RestoreApplications.plist to the device? It checks for errors though so it would have the restore process fail if the file could not be written, and it also prints Wrote RestoreApplications.plist if the file was created.

[intelfx]

@nikias I did not save the verbatim restore logs, but I distinctly remember the Wrote RestoreApplications.plist message in the beginning of a restore, yes.

[chris-fj]

Hello everyone! I am too finding this problem, with a command similar as the one specified in the first comment, Info.plist contains all the apps I want to be restored but the restore process doesn't trigger app reinstallation. Does anyone have an idea on what to do here?

[chris-fj]

For reference, I used plistutil to convert Status.plist to plain XML and this is the content of it...maybe it sheds some light

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>IsFullBackup</key>
        <false/>
        <key>Version</key>
        <string>3.3</string>
        <key>UUID</key>
        <string>xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx</string>
        <key>Date</key>
        <date>2022-05-20T21:33:10Z</date>
        <key>BackupState</key>
        <string>new</string>
        <key>SnapshotState</key>
        <string>finished</string>
</dict>
</plist>

[BehradJi]

Hi, I have the same problem here, on iOS 15.5 and latest git of libimobiledevice, after a backup with --full tag and restore that with --system and --settings tags, it does not trigger reinstallation of apps. But the interesting thing here is if I install all apps manually and try to restore with the same tags again it will trigger the reinstallation of old version of apps and their data will come back.

[chris-fj]

Bringing attention back on this

I tried a full backup again, in hopes of ditching iTunes and properly storing the backup instead of it being in a sketchy folder god knows where in my W*ndoze partition. I created from scratch a full backup of my iPhone XS running iOS 16.0 with the following command:

>> idevicebackup2 backup /run/media/xxx/backup/ios.bak

and the output was the following (I cut out the individual progress because it's irrelevant)

and the output was the following

Backup directory is "/run/media/xxx/backup/ios.bak"
Started "com.apple.mobilebackup2" service on port 59019.
Negotiated Protocol Version 2.1
Starting backup...
Backup will be encrypted.
Requesting backup from device...a
Full backup mode.
[=                                                 ]   0% Finished
Receiving files
[=                                                 ]   0% Finished/43.7 MB)       
Receiving files
...
[lots of output]
Sending '0000xxxx-00000000000000000/Status.plist' (189 Bytes)
Sending '0000xxxx-00000000000000000/Manifest.plist' (135.5 KB)
Sending '0000xxxx-00000000000000000/Manifest.db' (142.0 MB)
Received 87617 files from device.
Backup Successful.

after this, I wiped out my phone from its settings and tried to restore it with the following command:

idevicebackup2 -i -u '0000xxxx-00000000000000000' restore --system --settings /run/media/xxx/backup/ios.bak/

Backup directory is "/run/media/xxx/backup/ios.bak/"
This is an encrypted backup.
Enter backup password: ******************
Started "com.apple.mobilebackup2" service on port 49192.
Negotiated Protocol Version 2.1
Reading Info.plist from backup.
Starting Restore...
Restoring system files: Yes
Rebooting after restore: Yes
Don't copy backup: Yes
Preserve settings of device: No
Remove items that are not restored: No
Backup password: Yes
Wrote RestoreApplications.plist
Sending '0000xxxx-00000000000000000/Status.plist' (189 Bytes)
Sending '0000xxxx-00000000000000000/Manifest.plist' (135.5 KB)
Sending '0000xxxx-00000000000000000/Manifest.db' (142.0 MB)
Sending '0000xxxx-00000000000000000/Info.plist' (1.7 MB)
...
[==================================================] 100% Finished
The device should reboot now.
Restore Successful.

even though I specified the flags --system --settings, the restore output spits Preserve settings of device: No but regardless of that, after my device rebooted it seemed to keep the system settings (wallpaper, iOS widgets, etc...) but NO download of applications was triggered. I left it some time but to no avail so I tried a second restore with the same command on top of the previous backup, but I was hit with the "Insufficient space" error so I had to resort to the iTunes backup I did beforehand and my device was as fresh as before I started all of this.

There seems to be a problem with the application reinstall trigger after restore. If anyone needs any other test or output I could provide, let me know

[nikias]

Thanks for the feedback, I haven't tried in a while to restore a backup. I will have a look when I can. FYI --settings means it should restore the settings from the backup, and hence it will obviously say Preserve settings of device: No.

[windowalter]

can we restore setup setting from old backup with out reboot