Open kevinant2 opened 2 years ago
mexmer
commented
2 years ago if you see this error, there must have been some errors during backup already, either phone was not able to read file from device, or idevicebackup didn't save the file, but responded with error back to device (might be permission issue, or not enough free space ... but you should see that error).
manifest is generated by device, and it will not be generated in case of incomplete backup.
airwolf2026
commented
2 years ago May be you could delete some app or data on device, then try again.
This is the device backup service problem.
mexmer
commented
2 years ago yes that is also possibly, backup process (inside device) makes copy of files (at least opened ones) before sending them to computer, if there is not enough room in device, it will internally error, and not send file, but sometimes continue until end.
kevinant2
commented
2 years ago Hi all,
thanks for your responses.
This happens originally with the computer sleeping in the middle of the backup process sometimes ago. Then "Null" photo album related files were generated. Those "Null" files can be moved when filesystem is mounted but they can't be deleted and it appears to be quite weird that after backup interruptions from sleeping computer that Null files are generated.
Initially the phone would not able to upload any files (it was on iOS 14.6) and showing error that Null bytes were found in some files. After upgrading it to iOS15.3 the backup progress proceeds and it appears that almost all of files are uploaded before the error occurs. The Manifest.db and manifest.plist were not generated in the backup.
So I just wondering if the Manifest can be generated in some way or if those "Null" files can be removed.
Thanks again.
Kevin
mexmer
commented
2 years ago unfortunately, unless you fix data inside phone, nothing can be done.
manifest.plist and manifest.db are generated by phone (backupservice in phone), and it will not create them, until it's finished. manifest database contains files metadata as well as their encryption keys, and those are stored inside phone, and i don't know about any way how to get them (maybe from jailbroken phone, but i don't know about working jailbreak for ios 15.3 either).
kevinant2
commented
2 years ago Thanks mexmer,
It is weird and feels not quite acceptable just after the interruptions in the backup those Null files are generated that stuffed up the backup process of the phone. Do you know if there are any way to effectively delete those null byte files on the iphone? They are undeletable with current tools.
Thanks,
Kevin
mexmer
commented
2 years ago do you have in target folder also previous backup of phone? if so, then do backup to different folder, because normaly phone cheks for existing backup, and then only upload changed files. if there is no previous backup, it does full backup.
kevinant2
commented
2 years ago Thanks mexmer, I don't have a previous backup of this phone so it's doing full backup, therefore I am guessing it's touching those generated NULL files.
I've recorded iphone path and the SHA1 hash of the file but there are still gaps between that and the manifest file.
mexmer
commented
2 years ago can you post syslog? start on one console idevicesyslog, and redirect it's output to file, then on other console start idevicebackup. stop syslog, after it fails.
if you are concerned about privacy, just send me download link for log trough private message on github.
kevinant2
commented
2 years ago Thanks Mexmer.
The following is the syslog around the time when error occured. It appears that there's a "Backup Completed" message sent before the error is thrown out .
Hope this will gives some clue.
Oct 1 17:58:33 kernel[0]
mexmer
commented
2 years ago i did really hope, it will show some file name, but doesn't seems so, also log entry looks it has problem backing up itunes media database, it did send notification to itunes that it should copy media database to temporary folder, and after few tries it failed.
maybe itunes in phone is downloading something, no idea, it's blind shot, but put phone into airplane mode, then do backup.
kevinant2
commented
2 years ago Thanks mexmer, I have more complete log but it's more than half a million of lines. I've just tried to backup with the phone in the airplane mode but still with same MBErrorDomain/104. I've also managed to record iphone filepath, backup filepath and domain of backup files. The following is the log with the new backup. The log is as follows:
Oct 3 06:47:18 mediaanalysisd-service(PhotoLibraryServices)[47898]
mexmer
commented
2 years ago do you have any mac or windows pc with itunes, so you can try to do local backup on them?
kevinant2
commented
1 year ago Hi mexmer, I've tried to perform a backup with itunes / airplane mode. The backup files are there but the same issue still occurs and there are no manifest.db.
mexmer
commented
1 year ago if even desktop itunes fails, then definetly something is wrong either with database, or filesystem itself. does syslog show same multiple MBiTunesStorePlugin backingUpSQLiteFileCopyAtPath:temporaryPath message?
if so, then probly best would be to do icloud backup, prior doing any experiments on filesystem, but then i would mount device filesystem and delete photos database, and do idevicebackup, if this doesn't work, second step would be deleting itunes media database. but tbh. deleting anything from phone filesystem direcly can also cause another bunch of issues, and also data loss (by deleting photos database, you will loose albums, by deleting itunes media database, you will loose list of synced media, and you might have then orphaned media in itunes folder, that will not show up, but will take space). it's not a thing i would recommend doing at all.
samuelsadok
commented
6 months ago As others have pointed out, this is not a libimobiledevice issue. I ran into the same issue with a normal backup in macOS Finder. But since I came across this thread while investigating, I figured I'd add my findings here in case it helps anyone else.
Your device needs to be jailbroken for this.
In my case, on a iOS 15.8.2 device, I could see a handful of messages like these in the syslog:
BackupAgent2 Failed to upload <private> to <private> in batch 679: <NSError:0x281101f50(MBErrorDomain:100) - {NSLocalizedDescription = <private>;}>
BackupAgent2 Failed to upload <private> to <private> in batch 707: <NSError:0x281120cc0(MBErrorDomain:100) - {NSLocalizedDescription = <private>;}>
...
BackupAgent2 Failed to upload after X batches [...]The file names are redacted behind <private> but could be revealed (in a second run) thanks to @EthanArbuckle's method described here: https://github.com/EthanArbuckle/unredact-private-os_logs
378 error 16:44:14.650227+0100 BackupAgent2 Failed to upload /.ba/mobile/Media/DCIM/108APPLE/IMG_8720.HEIC to ea75b4bce2c5c15387720dc3202373ac2dae53c5/Snapshot/14/1477d34c9071bb8760c8d25d182563029539a424 in batch 679: <NSError:0x281101f50(MBErrorDomain:100) - {NSLocalizedDescription = "Input/output error (5)";}>
378 error 16:47:36.416450+0100 BackupAgent2 Failed to upload /.ba/mobile/Media/DCIM/119APPLE/IMG_9294.JPG to ea75b4bce2c5c15387720dc3202373ac2dae53c5/Snapshot/74/74bf5698e8ef46ea869a31a9579f65d241f5acd1 in batch 707: <NSError:0x281120cc0(MBErrorDomain:100) - {NSLocalizedDescription = "Input/output error (5)";}>
...This is just an exert, the bad files also included application data, not only media files.
So I ran a backup while recording the syslog to get a list of all bad files. (I don't know why they were bad, but this device had update problems before so maybe that was related.) Then I moved all bad files to a stash location. (I prefer moving over deleting, in case I need to revert it.) Note that /.ba/ is a snapshot mount (probably) and the original files are in /private/var/.
mkdir /private/var/backup_quarantine
mv /private/var/mobile/Media/DCIM/108APPLE/IMG_8720.HEIC /private/var/backup_quarantine/
mv /private/var/mobile/Media/DCIM/119APPLE/IMG_9294.JPG /private/var/backup_quarantine/
...And that's it, after that the backup completed successfully!
Just for reference, here's the syslog of a good backup (filter: Errors&Warnings, "BackupAgent2"). This was recorded on a different, normally functioning iOS 15.8 device. This helped me distinguish true errors from red herrings:
19819 error 16:04:10.832951+0100 BackupAgent2 [ICUserIdentityStore] - initializing account histories with <private>
19819 error 16:04:10.878356+0100 BackupAgent2 Missing books entry.
19819 error 16:04:55.213037+0100 BackupAgent2 unmount failed at /.ba: [22: Invalid argument]
19819 error 16:04:56.412266+0100 BackupAgent2 API call with NULL database connection pointer
19819 error 16:04:56.412297+0100 BackupAgent2 misuse at line 176255 of [9ff244ce07]
19819 error 16:04:56.412322+0100 BackupAgent2 can't clear busy handler
19819 error 16:04:57.417016+0100 BackupAgent2 open_dprotected_np failed at <private>: [2: No such file or directory]
19819 error 16:04:57.417727+0100 BackupAgent2 open_dprotected_np failed at <private>: [2: No such file or directory]
[repeats several times...]
19819 error 16:06:46.756053+0100 BackupAgent2 cannot open file at line 37488 of [9ff244ce07]
19819 error 16:06:52.266576+0100 BackupAgent2 cannot open file at line 37488 of [9ff244ce07]
19819 error 16:07:03.331610+0100 BackupAgent2 no such collation sequence: IDBKEY in "CREATE TABLE Records (objectStoreID INTEGER NOT NULL ON CONFLICT FAIL, key TEXT COLLATE IDBKEY NOT NULL ON CONFLICT FAIL, value NOT NULL ON CONFLICT FAIL, recordID INTEGER PRIMARY KEY)"
19819 error 16:07:03.332194+0100 BackupAgent2 _sqlite3_db_copy_compact failed at <private>: 1/0x1
19819 error 16:07:03.332593+0100 BackupAgent2 Failed to compact SQLite database at <private>: Error Domain=MBErrorDomain Code=16 UserInfo={NSFilePath=<private>, NSLocalizedDescription=<private>}
[group of 3 warnings occurs several times]
19819 error 16:08:40.692804+0100 BackupAgent2 unmount failed at /.ba: [22: Invalid argument]
19819 error 16:08:40.716527+0100 BackupAgent2 unmount failed at /.ba: [22: Invalid argument]Since there were some questions about these files:
On a jailbroken device you can see both files while the backup is running (UUID in the path may vary):
> ls -lah /private/var/root/Library/Caches/Backup/BC2368F2-E882-480B-A338-FD1167735347/
total 396384
drwxr-xr-x 6 root wheel 192B Mar 21 16:24 .
drwxr-xr-x 13 root wheel 416B Mar 21 16:21 ..
-rw-r--r-- 1 root wheel 187M Mar 21 16:51 Manifest.db
-rw-r--r-- 1 root wheel 64K Mar 21 16:22 Manifest.db-shm
-rw-r--r-- 1 root wheel 1.2M Mar 21 16:52 Manifest.db-wal
-rw-r--r-- 1 root wheel 193K Mar 21 16:24 Manifest.plistThis folder is deleted at the end of the backup. So an alternative approach to what I described above could be to patch /usr/libexec/BackupAgent2 or use some filesystem trickery to prevent deletion at the end of a backup and then transfer them manually (e.g. via SSH). You'd probably need to use additional tools to repair the resulting backup.
Hi All,
Just would like to know if this is solvable or if there are any ways around this. I've encountered backup issues as the following:
If there are no solution, are there any way to get Manifest.plist and Manifest.db generated?
Thank you very much for your help, the following is the output from the libimobiledevice
Kevin
Receiving files [==================================================] 100% (803.2 KB/803.2 KB) [==================================================] 99% Finished Receiving files [==================================================] 99% Finished/41.0 MB) Receiving files [==================================================] 100% (376.3 KB/376.3 KB) [==================================================] 99% Finished ErrorCode 104: Multiple errors uploading files (MBErrorDomain/104) Received 193946 files from device. Backup Failed (Error Code 104).