How to extract adb backups made with the --twrp option?

[ghost]

I have a Samsung Galaxy S2 GT-I9100 smartphone with LineageOS and TWRP. Every week I make a backup with the following command:

adb backup -f twrp-20170322.ab --twrp boot data system

I may optionally use the --compress option, too.

Is there a way to extract the twrp-20170322.ab backup file with stardard GNU/Linux command line tools? There are a number of tutorials online that use dd/tar/openssl, but they don't work when using the --twrp option.

I previously asked this question on XDA Developers and on Stack Exchange without success.

[ClayF]

Looking for a way to do this as well.

[jontis]

Check your file: $ head backup.ab

Twrp has added some things in the beginning. Skipping those with: $ dd if=backup.ab bs=512 skip=3 | head

Got me to a semiworking extraction.

$ dd if=backup.ab bs=512 skip=3 | tar -x

[heartcomputers]

I also am having trouble with this

[tofurky]

just saw this show up in the comment section of the stackexchange link posted in the issue (on june 9th): https://github.com/arifogel/twrpabx

it seems to work as advertised. the generated output file causes lots of errors like "tar: Malformed extended header: missing equal sign" when listing/extracting with GNU tar. that's an artifact of the twrp "tar" format though, i think. i just tested and it seems to extract with tar OK even with those error messages. i did not test with a compressed backup.

to build it on linux:

autoreconf -i
./configure
make

thanks @arifogel

[arifogel]

@tofurky I'm glad you were able to make use of my tool. FWIW I don't recall getting any tar complaints when I ran at least on my stuff. I'm open to refining the project if you would be willing to toss me some test cases/issues.

[heartcomputers]

@tofurky thanks for sharing this. @arifogel thanks for putting this little program together. Unfortunately, when pointing it at my .ab file (uncompressed), I get "Corrupt start of data header". It outputs a system.ext4.win file which my archive manager complains is corrupt but extraction with the tar command seems to go smoothly. Unfortunately it stops at the system partition.

[AlD]

The reason for the tar errors is the weird block header format that TWRP uses. Unless the format changed in the last 2 years, this small Python script will convert a TWRP .ab backup to .tar: https://gist.github.com/AlD/cb443fe4ba152e1de8a42fa72721552b

[CoREse]

The reason for the tar errors is the weird block header format that TWRP uses. Unless the format changed in the last 2 years, this small Python script will convert a TWRP .ab backup to .tar: https://gist.github.com/AlD/cb443fe4ba152e1de8a42fa72721552b

this simple script helps a lot! thank you! and of course, it works!

[unilock]

Here's how I extracted my multi-partition ADB / TWRP backup when nothing else worked:

• Opened "backup.ab" in HxD (or other hex editor)
• Searched for "data.ext4.win"
• Looked for "TWRP....twfilename" shortly before the occurence of "data.ext4.win"
• Placed cursor directly before the occurence of "TWRP....twfilename"
• Shift-clicked (selected) the location directly before the first occurence of "TWRP....filename" in "backup.ab" (near "system.ext4.win")
• Deleted all selected data
• Saved file as "backup2.ab"
• Used twrp2tar (linked above) on "backup2.ab" to get "backup2.tar" -- this is the data partition

Hopefully these shoddy, hastily written instructions can help someone else in the future.

[Jindey]

I have a TWRP backup of my data partition @unilock what should I select from the begin to the first system.ext4.win??

[ChrizZz90]

Is there a new way to extract the data? I tried @unilock guide and also @AlD python script but without success. I have a backup.ab file with several partitions. In both ways I get a tar file but inside the tar file is just the backup.ab and I can't unpack it due to a checksum error.

[unilock]

@Jindey @ChrizZz90 I made a short video showing my process in more detail. You'll need to download Python 2 and twrp2tar, then open your backup.ab file in a hex editor, e.g. HxD.

https://user-images.githubusercontent.com/11802133/136434827-75cacce1-dd25-4e63-b1f7-c98c666c1516.mp4

[ChrizZz90]

@unilock thanks for the video, I really made a mistake and selected the wrong block. But I may have strange backups, but I have "TWRP....twfilename" only once in my *.ab files. In my understanding, I wouldn't remove anything in this case? But this will lead to tar issues and the Python script runs in a never-ending loop. I have 2 backups, one has only the data partition, the other one system, boot, recovery, vendor and data. During the conversion to a tar file, I don't have the case of "TWRP\x00\x00\x00\x00".

[unilock]

@ChrizZz90 If you have a backup containing only the data partition, I believe you should only need to run twrp2tar, without deleting anything in the backup. But it sounds like you already tried that, so I'm not sure where to go from there.

[ChrizZz90]

indeed, I tried that already. Same today, just with python3 and some minor changes that print is working with python3. Converting my 30gb backup takes 20min but the script is in an endless loop. I need to stop the script manually but the tar file is corrupt after that. I just have a lib folder and the AndroidManifest.xml in the tar file.

[sandros87]

I have TWRP adb backup with all partitions but I only have two TWRP....twfilename for vendor and data however when I restore this update it also contains boot efs and (I hope) system. Why do you think is that? I would like to extract system or at least make sure it's actually there!

[bigbiff]

The intent of adb backup, if you want to restore individual partitions, you should backup only that partition over adb backup and name the file appropriately.

[unilock]

@sandros87 I'm not familiar with the way TWRP creates backups, but perhaps boot (ramdisk?) and efs ("Encrypted File System" - stores unique device identifiers) are stored differently than the ext4 partitions? Not sure about system though, that should be stored as ext4... maybe your backup doesn't include it?

In any case, you could do a text search for "system" in your TWRP backup file, or try restoring the backup on a spare phone with TWRP.

[jerrykan]

I wrote a short python script to extract files/image from a mutli partition TWRP backup file. It isn't especially robust, so there are probably backup files that it will fail on, but it might be a good starting point for someone else who comes across this thread looking for help.

https://gist.github.com/jerrykan/b3b2f7f43b5e1d4d5b5198748be16bb2

The extracted files were just TAR files.