search

YuvrajRaghuvanshiS / WhatsApp-Key-Database-Extractor

The most advanced and complete solution for extracting WhatsApp key/DB from package directory (/data/data/com.whatsapp) without root access.
MIT License
464 stars 60 forks source link

Error : whatsapp.ab size 549 bytes/ could not extract whatsapp.tar #16

Closed Benji005 closed 3 years ago

Benji005 commented 3 years ago

Describe the bug No extracted file

To Reproduce Steps to reproduce the behavior:

  1. run 'wa_kdbe.py'
  2. Enter 'y'
  3. See error:
  4. bin\tar.exe: Record size = 2 blocks bin\tar.exe: apps/com.whatsapp/f/key: Not found in archive bin\tar.exe: Error exit delayed from previous errors [WinError 3] The system cannot find the path specified: 'tmp/apps/com.whatsapp/f/key' -> 'extracted/mine/key' Cleaning up tmp folder...

Expected behavior File in extracted folder

Screenshots If applicable, add screenshots to help explain your problem.

Terminal output bin\tar.exe: Record size = 2 blocks bin\tar.exe: apps/com.whatsapp/f/key: Not found in archive bin\tar.exe: Error exit delayed from previous errors [WinError 3] The system cannot find the path specified: 'tmp/apps/com.whatsapp/f/key' -> 'extracted/mine/key' Cleaning up tmp folder...

Desktop (please complete the following information):

Smartphone (please complete the following information):

Additional context Add any other context about the problem here.

roberts011 commented 3 years ago

What WhatsApp version do you use? Also, did you download the prerequsities (python 3.x, latest java, adb)?

YuvrajRaghuvanshiS commented 3 years ago

It happens sometimes when adb can't create whatsapp.ab file properly. The process of creating backup, you must have noticed that the process completed quickly. Try re running the script.

YuvrajRaghuvanshiS commented 3 years ago

Send complete output not just error.

Benji005 commented 3 years ago

I already tried... twice I will try again and copy paste the complete error for you

Benji005 commented 3 years ago
Failure [INSTALL_FAILED_USER_RESTRICTED: Install canceled by user] - Found Java installed on system. - Please read above instructions carefully. Continue? (default y) : y - Connected to Redmi Note 5 - WhatsApp V2.21.1.13 installed on device - Found legacy WhatsApp V2.11.431 apk in helpers/ folder - Backing up WhatsApp 2.21.1.13 apk, the one installed on device to tmp/WhatsAppbackup.apk - 5542 KB/s (31608405 bytes in 5.569s) - Apk backup complete. - Uninstalling WhatsApp, skipping data. - Success - Uninstalled. - Installing legacy WhatsApp V2.11.431, hold tight now. - 4295 KB/s (18329558 bytes in 4.167s) - Failure [INSTALL_FAILED_USER_RESTRICTED: Install canceled by user] - Installation Complete. - Backing up WhatsApp data as tmp/whatsapp.ab. May take time, don't panic. - Now unlock your device and confirm the backup operation. - Done backing up data. - Reinstallting original WhatsApp. - 4791 KB/s (31608405 bytes in 6.441s) - Failure [INSTALL_FAILED_USER_RESTRICTED: Install canceled by user] - Our work with device has finished, it is safe to remove it now. - Found whatsapp.ab in tmp folder. Continuing - Enter a reference name for this user. : Bkp - Enter same password which you entered on device when prompted earlier. : 12345 - Successfully 'fluffed' tmp/whatsapp.ab tmp/whatsapp.tar - Taking out main files in tmp/ folder temporaily. - bin\tar.exe: Record size = 2 blocks - bin\tar.exe: apps/com.whatsapp/f/key: Not found in archive - bin\tar.exe: Error exit delayed from previous errors - [WinError 3] The system cannot find the path specified: 'tmp/apps/com.whatsapp/f/key' -> 'extracted/Bkp/key' - Cleaning up tmp folder... -
YuvrajRaghuvanshiS commented 3 years ago

Failure [INSTALL_FAILED_USER_RESTRICTED: Install canceled by user]

I believe you are using MIUI. MIUI restricts apk installation via adb. You have to allow that. Here's a quick guide

You might need to add MI account to your device for that

Benji005 commented 3 years ago
Failure [INSTALL_FAILED_VERSION_DOWNGRADE] - Found Java installed on system. - Please read above instructions carefully. Continue? (default y) : y - Connected to Redmi Note 5 - WhatsApp V2.21.1.13 installed on device - Found legacy WhatsApp V2.11.431 apk in helpers/ folder - Backing up WhatsApp 2.21.1.13 apk, the one installed on device to tmp/WhatsAppbackup.apk - 5294 KB/s (31608405 bytes in 5.829s) - Apk backup complete. - Uninstalling WhatsApp, skipping data. - Success - Uninstalled. - Installing legacy WhatsApp V2.11.431, hold tight now. - 4875 KB/s (18329558 bytes in 3.671s) - **Failure [INSTALL_FAILED_VERSION_DOWNGRADE]** - Installation Complete. - Backing up WhatsApp data as tmp/whatsapp.ab. May take time, don't panic. - Now unlock your device and confirm the backup operation. - Done backing up data. - Reinstallting original WhatsApp. - 4679 KB/s (31608405 bytes in 6.596s) - Success - Our work with device has finished, it is safe to remove it now. - Found whatsapp.ab in tmp folder. Continuing
YuvrajRaghuvanshiS commented 3 years ago

Try this and send output.

In same terminal copy paste following commands.

git pull origin git checkout issues/16 python wa_kdbe.py

Benji005 commented 3 years ago

What do you mean by the same terminal? I dont get you!

YuvrajRaghuvanshiS commented 3 years ago

I meant in same directory. Where you have this project. Like this image

Benji005 commented 3 years ago

What WhatsApp version do you use? My version 2.21.1.13 Also, did you download the prerequsities (python 3.x, latest java, adb)?

Yes

YuvrajRaghuvanshiS commented 3 years ago

If you don't have git installed on system you can do this.

Go to : https://github.com/YuvrajRaghuvanshiS/WhatsApp-Key-Database-Extractor/tree/issues/16

Download project zip : ![image](https://user-images.githubusercontent.com/52948878/106122850-30a82980-617f-11eb-9d18-bed76f8b02ee.png) Unzip and continue regularly.
Benji005 commented 3 years ago
TypeError: can only concatenate str (not "int") to str - Found Java installed on system. - Please read above instructions carefully. Continue? (default y) : y - Connected to Redmi Note 5 - WhatsApp V2.21.1.13 installed on device - Downloading legacy WhatsApp V2.11.431 to helpers folder - 100% [........................................................................] 18329558 / 18329558Backing up WhatsApp 2.21.1.13 apk, the one installed on device to tmp/WhatsAppbackup.apk - 4729 KB/s (31608405 bytes in 6.525s) - Apk backup complete. - Uninstalling WhatsApp, skipping data. - Success - Uninstalled. - Installing legacy WhatsApp V2.11.431, hold tight now. - Traceback (most recent call last): - File "F:\Backups\WhatsApp-Key-Database-Extractor-issues-16\wa_kdbe.py", line 148, in - main() - File "F:\Backups\WhatsApp-Key-Database-Extractor-issues-16\wa_kdbe.py", line 42, in main - USBMode() - File "F:\Backups\WhatsApp-Key-Database-Extractor-issues-16\wa_kdbe.py", line 142, in USBMode - RealDeal(SDKVersion, WhatsAppapkPath, versionName) if ACReturnCode==1 else Exit() - File "F:\Backups\WhatsApp-Key-Database-Extractor-issues-16\wa_kdbe.py", line 95, in RealDeal - InstallLegacy(SDKVersion) - File "F:\Backups\WhatsApp-Key-Database-Extractor-issues-16\wa_kdbe.py", line 88, in InstallLegacy - CustomPrint('SDK Version is : ' + SDKVersion) - TypeError: can only concatenate str (not "int") to str
Benji005 commented 3 years ago

I think since the legacy whatsapp version is older than the original whatsapp version installed in my device the code is not working.

YuvrajRaghuvanshiS commented 3 years ago

Yes you are thinking right. I'm sorry for this error above it was bad code on my side and not related to your issue. Do a quick git pull origin git checkout issue/16 python wa_kdbe.py

Or re-download zip from GitHub whichever way you're doing this.

Benji005 commented 3 years ago
- SDK Version is : 27 - 4452 KB/s (18329558 bytes in 4.020s) - Failure [INSTALL_FAILED_VERSION_DOWNGRADE] - Found Java installed on system. - Please read above instructions carefully. Continue? (default y) : y - Connected to Redmi Note 5 - WhatsApp V2.21.1.13 installed on device - Found legacy WhatsApp V2.11.431 apk in helpers/ folder - Backing up WhatsApp 2.21.1.13 apk, the one installed on device to tmp/WhatsAppbackup.apk - Folder tmp/ already exists. - 4410 KB/s (31608405 bytes in 6.999s) - Apk backup complete. - Uninstalling WhatsApp, skipping data. - Success - Uninstalled. - Installing legacy WhatsApp V2.11.431, hold tight now. - **SDK Version is : 27** - 4452 KB/s (18329558 bytes in 4.020s) - **Failure [INSTALL_FAILED_VERSION_DOWNGRADE]** - Installation Complete. - Backing up WhatsApp data as tmp/whatsapp.ab. May take time, don't panic. - Now unlock your device and confirm the backup operation. - Done backing up data. - Reinstallting original WhatsApp. - 5021 KB/s (31608405 bytes in 6.147s) - Success - Our work with device has finished, it is safe to remove it now. - Found whatsapp.ab in tmp folder. Continuing - Enter a reference name for this user. : issue16 - Enter same password which you entered on device when prompted earlier. : - Successfully 'fluffed' tmp/whatsapp.ab tmp/whatsapp.tar - Taking out main files in tmp/ folder temporaily. - bin\tar.exe: Record size = 2 blocks - bin\tar.exe: apps/com.whatsapp/f/key: Not found in archive - bin\tar.exe: Error exit delayed from previous errors - [WinError 3] The system cannot find the path specified: 'tmp/apps/com.whatsapp/f/key' -> 'extracted/issue16/key' - Cleaning up tmp folder... -
Benji005 commented 3 years ago

Yes you are thinking right. I'm sorry for this error above it was bad code on my side and not related to your issue. Do a quick git pull origin git checkout issue/16 python wa_kdbe.py

Or re-download zip from GitHub whichever way you're doing this.

Yeah i got it! But still the issue persists!

YuvrajRaghuvanshiS commented 3 years ago

It is saying 'success' when uninstalling whatsapp so downgrading error is not because of apk but because of data (which is not deleted while uninstalling original whatsapp). And I believe that is doing something in not installing legacy apk. I need some time to research into this.

I have used '-d' flag while installing apk adb install -r -d legacy whatsapp which explicitly tells device to allow downgrading of apk

Benji005 commented 3 years ago

Okay... sure take your time.

I will check back later!

YuvrajRaghuvanshiS commented 3 years ago

In the meantime you can try this however I'm no longer maintaining this fork : https://github.com/YuvrajRaghuvanshiS/WhatsDump

Benji005 commented 3 years ago

In the meantime you can try this however I'm no longer maintaining this fork : https://github.com/YuvrajRaghuvanshiS/WhatsDump

I have tried the same and there is syntax error in the code.

Refer below: File "F:\Backups\WhatsDump-master\whatsdump.py", line 228 except WaException, e: ^ SyntaxError: invalid syntax

roberts011 commented 3 years ago

WhatsDump only runs with python 2.7

YuvrajRaghuvanshiS commented 3 years ago

Thank you @rschuppan for pointing that out. Sorry I missed that. You need to download python 2.7.x and run it with python2 whatsdump.py

Benji005 commented 3 years ago

Then I better wait for you to update this file itself. I have no rush to extract the key. Or else i will try it later

Anyways thanks! @YuvrajRaghuvanshiS @rschuppan

roberts011 commented 3 years ago

No prob, spent like half a day figuring that out when I tried WhatsDump the first time. Also, I think I remember that the java references are broken... @YuvrajRaghuvanshiS PLEASE add a note pointing that (python 2.7 thing) out on the readme of your WhatsDump version

YuvrajRaghuvanshiS commented 3 years ago

Then I better wait for you to update this file itself. I have no rush to extract the key. Or else i will try it later

Anyways thanks! @YuvrajRaghuvanshiS @rschuppan

This is not as easy as changing few lines in code.

Benji005 commented 3 years ago

Then I better wait for you to update this file itself. I have no rush to extract the key. Or else i will try it later Anyways thanks! @YuvrajRaghuvanshiS @rschuppan

This is not as easy as changing few lines in code.

Yeah... Okay. No problem!

YuvrajRaghuvanshiS commented 3 years ago

This method does not work work on every device. Recently a friend on mine faced issues on his Samsung device. That's why I'm thinking of re-working that fork of WhatsDump again, https://github.com/YuvrajRaghuvanshiS/WhatsDump/tree/python3Implementation

roberts011 commented 3 years ago

Problem with that is, the core concept of WhatsDump is faulty. The key spawned by WhatsApp on the emulator is NOT the same key thats on your original / real phone, and therefore cannot be used. I had to find this out the hard way, you remember by Nox-HowTo over at Signal community.

YuvrajRaghuvanshiS commented 3 years ago

Well we don't need key remember? Just msgstore.db is enough for us.

roberts011 commented 3 years ago

Yes, but how do you want to decrypt a msgstore without the key? The key one needs for their original msgstore on their phone is sadly only on this very phone.

YuvrajRaghuvanshiS commented 3 years ago

I don't need to decrypt that. Fortunately for us android emulator is a rooted device and we can just grab it from /data/data/com.whatsapp directory.

roberts011 commented 3 years ago

Since when do we not need to decrypt the msgstore? The key generated on the emulator does not fit the msgstore file on your real phone.

YuvrajRaghuvanshiS commented 3 years ago

Since when do we not need to decrypt the msgstore?

It is already in decrypted form in all phones in /data/data/com.whatsapp directory and emulator is rooted by default we can just go there and fetch it.

The key generated on the emulator does not fit the msgstore file on your real phone.

Well that's a valid point. But again decrypting that original msgstore.db.crypt12 file from phone with key is also not a easy process

roberts011 commented 3 years ago

It is already in decrypted form in all phones in /data/data/com.whatsapp directory and emulator is rooted by default we can just go there and fetch it.

Wait, out of impulse I would guess that this would be way easier? Couldn't we also pull that file from the generated backup?

But how would the emulator get the backup? Because I tried that, and the rooted emulator didn't have G-Drive, because Googles developer site states you can either have an unrooted emulator with Google Services or a rooted device without Google services..

YuvrajRaghuvanshiS commented 3 years ago

You do not need to have GDrive backup on emulator. Get encrypted backup from original phone /storage/emulated/0/WhatsApp/Databases directory then copy it at same location in emulator, install whatsapp, register and restore this backup in emulator. Then un-encrypted backup will be in /data/data/com.whatsapp directory.

roberts011 commented 3 years ago

Tried that, didn't work. The emulator-WhatsApp doesn't detect it.

YuvrajRaghuvanshiS commented 3 years ago

Does not detect what?

roberts011 commented 3 years ago

The backup file (from the original phone) that was copied into the emulator

YuvrajRaghuvanshiS commented 3 years ago

MarcoG3's WhatsDump works on this same concept. I did not tried this manually but this works. There must have been some issues with directory naming.

roberts011 commented 3 years ago

Well, I hope for you it works. Good luck! Also, I'd propose to move the whole "WhatsDump" part of this issue into a new one, or a discussion

YuvrajRaghuvanshiS commented 3 years ago

Thanks. Yeah you may do that on my fork of MarcoG3's WhatsDump. But I am unavailable for a month bcz of university exams.

YuvrajRaghuvanshiS commented 3 years ago

@Benji005 Do a git pull origin, git checkout issues/16, python wa_kdbe.py --allow-reboot

Benji005 commented 3 years ago

[10:36:37.180030] Please read above instructions carefully ↑ . Continue? (default y) : y [10:36:41.771131] Connected to Redmi Note 5 [10:36:46.629350] WhatsApp V2.21.1.13 installed on device [10:36:46.641983] Found legacy WhatsApp V2.11.431 apk in helpers/ folder [10:36:46.899739] Backing up WhatsApp 2.21.1.13 apk, the one installed on device to tmp/WhatsAppbackup.apk /data/app/com.whatsapp-4X_lZLkIbnuXT1Y_RshZLA==/base.apk: ...le pulled, 0 skipped. 18.2 MB/s (31608405 bytes in 1.653s) [10:36:48.989061] Apk backup complete. [10:36:48.989342] Uninstalling WhatsApp, skipping data. Success [10:36:51.493841] Uninstalled.

[10:36:51.499154] Rebooting device, please wait. [10:36:54.708709] Waiting for device... [10:36:59.783296] Waiting for device... [10:37:05.645882] Waiting for device... [10:37:13.824827] Waiting for device... [10:37:18.916031] Waiting for device... [10:37:24.042961] Waiting for device... [10:37:29.177299] Waiting for device... [10:37:34.492776] Waiting for device... [10:37:40.089487] Waiting for device... [10:37:45.796230] Waiting for device... [10:37:51.097883] Press any key after unlocking device. [10:38:01.409062] Installing legacy WhatsApp V2.11.431, hold tight now. Performing Streamed Install Success [10:38:48.904071] Installation Complete. [10:38:48.906732] Backing up WhatsApp data as tmp/whatsapp.ab. May take time, don't panic. WARNING: adb backup is deprecated and may be removed in a future release Now unlock your device and confirm the backup operation... [10:39:06.403593] Done backing up data. [10:39:06.406659] Reinstallting original WhatsApp. Performing Streamed Install Success

[10:39:29.378263] Our work with device has finished, it is safe to remove it now.

[10:39:29.381692] Found whatsapp.ab in tmp folder. Continuing [10:39:29.383684] Enter a reference name for this user. : Wa_bkp [10:39:52.753719] Enter same password which you entered on device when prompted earlier. : 321 [10:40:02.020115] Successfully 'fluffed' tmp/whatsapp.ab tmp/whatsapp.tar [10:40:02.020115] Folder extracted/ already exists. [10:40:02.021282] Taking out main files in tmp/ folder temporaily. bin\tar.exe: Record size = 2 blocks bin\tar.exe: apps/com.whatsapp/f/key: Not found in archive bin\tar.exe: Error exit delayed from previous errors [10:40:02.246874] [WinError 3] The system cannot find the path specified: 'tmp/apps/com.whatsapp/f/key' -> 'extracted/Wa_bkp/key' [10:40:02.254905] Cleaning up tmp folder...

@YuvrajRaghuvanshiS Still could not extract the key file!

YuvrajRaghuvanshiS commented 3 years ago

[10:38:01.409062] Installing legacy WhatsApp V2.11.431, hold tight now. Performing Streamed Install Success [10:38:48.904071] Installation Complete.

Breaking down the problems that's solved [INSTALL_FAILED_VERSION_DOWNGRADE]

[10:38:48.906732] Backing up WhatsApp data as tmp/whatsapp.ab. May take time, don't panic. WARNING: adb backup is deprecated and may be removed in a future release Now unlock your device and confirm the backup operation... [10:39:06.403593] Done backing up data.

The time difference between starting of backup creation and completion of that is almost 20 seconds so I believe it went just well. But just to be sure I have added size message to confirm that. You again need to pull origin and re-run this.

Benji005 commented 3 years ago

[12:58:50.333857] Found Java installed on system.

[12:58:50.342398] Please read above instructions carefully ↑ . Continue? (default y) : y [12:58:54.539190] Connected to Redmi Note 5 [12:58:56.065749] WhatsApp V2.21.1.13 installed on device [12:58:56.074718] Downloading legacy WhatsApp V2.11.431 to helpers folder 100% [........................................................................] 18329558 / 18329558[12:59:07.332704] Backing up WhatsApp 2.21.1.13 apk, the one installed on device to tmp/WhatsAppbackup.apk /data/app/com.whatsapp-lKPW7s4TgzrIj40NeMnJ1w==/base.apk: ...le pulled, 0 skipped. 17.4 MB/s (31608405 bytes in 1.732s) [12:59:09.218438] Apk backup complete. [12:59:09.221226] Uninstalling WhatsApp, skipping data. Success [12:59:11.474888] Uninstalled.

[12:59:11.484265] Rebooting device, please wait. [12:59:14.698586] Waiting for device... [12:59:19.860142] Waiting for device... [12:59:24.988653] Waiting for device... [12:59:30.119685] Waiting for device... [12:59:35.256152] Waiting for device... [12:59:40.371789] Waiting for device... [12:59:45.496130] Waiting for device... [12:59:50.634509] Waiting for device... [12:59:55.753351] Waiting for device... [13:00:00.883862] Waiting for device... [13:00:06.012327] Waiting for device... [13:00:11.166389] Waiting for device... [13:00:16.280961] Waiting for device... [13:00:21.471115] Waiting for device... [13:00:26.669619] Press any key after unlocking device. [13:00:28.306370] Installing legacy WhatsApp V2.11.431, hold tight now. Performing Streamed Install Success [13:00:51.241771] Installation Complete. [13:00:51.242420] Backing up WhatsApp data as tmp/whatsapp.ab. May take time, don't panic. WARNING: adb backup is deprecated and may be removed in a future release Now unlock your device and confirm the backup operation... [13:01:04.278295] Done backing up data. Size : 549 bytes. [13:01:04.278295] Reinstallting original WhatsApp. Performing Streamed Install Success

[13:01:27.983168] Our work with device has finished, it is safe to remove it now.

[13:01:27.986437] Found whatsapp.ab in tmp folder. Continuing... Size : 549 bytes. [13:01:27.990180] Enter a reference name for this user. : New_bkp [13:01:38.801163] Enter same password which you entered on device when prompted earlier. : 1234 [13:01:42.894379] Successfully 'fluffed' tmp/whatsapp.ab tmp/whatsapp.tar. Size : 549 bytes. [13:01:42.894379] Folder extracted/ already exists. [13:01:42.896398] Taking out main files in tmp/ folder temporaily. bin\tar.exe: Record size = 2 blocks bin\tar.exe: apps/com.whatsapp/f/key: Not found in archive bin\tar.exe: Error exit delayed from previous errors [13:01:43.017870] [WinError 3] The system cannot find the path specified: 'tmp/apps/com.whatsapp/f/key' -> 'extracted/New_bkp/key' [13:01:43.021766] Cleaning up tmp folder...

Yes. The Error is solved. I have re-run the new file again. But still i could not extract the key file.

YuvrajRaghuvanshiS commented 3 years ago

[13:01:04.278295] Done backing up data. Size : 549 bytes.

This is happening on some devices and I'm still figuring out why. Backup size goes as high as several hundred MBs but on some device it does not get created in the first place. Like your case. This 'whatsapp.ab' file contains only file header and nothing useful for us.

@Tkd-Alex what are your thoughts on this?

YuvrajRaghuvanshiS commented 3 years ago

From https://docs.j7k6.org/android-adb-backup-41-bytes-549-bytes-bug-fix/

Appearantly the adb backup command stopped working for older Android devices in recent versions of the Android Debug Bridge. When invoking a backup with adb, the backup process is started but stops immediately, leaving a local backup.ab file with a filesize of only 49 bytes (549 bytes when the backup is encrypted with a password). This bug is well known, but hasn’t been fixed at least until recently (version 1.0.39). To fix this behaviour, download the last working version of the adb executable (1.0.31) and invoke the backup with this. The executables for all platforms can be downloaded at Mozilla.

I am going to try this one as well. I will let you know after implementing.

YuvrajRaghuvanshiS commented 3 years ago

Sync your repo with origin and then run with --alternate-adb. : python wa_kdbe.py --allow-reboot --alternate-adb

Benji005 commented 3 years ago

[20:04:57.745505] Please read above instructions carefully ↑ . Continue? (default y) : y [20:05:01.707322] Connected to Redmi Note 5 [20:05:03.293326] WhatsApp V2.21.1.13 installed on device [20:05:03.293326] Found legacy WhatsApp V2.11.431 apk in helpers/ folder adb server is out of date. killing...

[20:05:08.576856] Rebooting device, please wait. error: device not found [20:05:08.661452] Waiting for device... [20:05:13.756271] Waiting for device... [20:05:18.826525] Waiting for device... [20:05:23.872690] Waiting for device... [20:05:28.943376] Waiting for device...

Error shown as device not found, but device is connected with the PC.