search

KnugiHK / WhatsApp-Chat-Exporter

A customizable Android and iOS/iPadOS WhatsApp database parser that will give you the history of your WhatsApp conversations in HTML and JSON. Android Backup Crypt12, Crypt14, Crypt15, and new schema supported.
https://wts.knugi.dev/
MIT License
518 stars 76 forks source link

Android 12: Export: wa.db - Contacts are not exported. #63

Open sietsevdschoot opened 9 months ago

sietsevdschoot commented 9 months ago

I have a OnePlus Android 12 Phone.

Using WhatsApp-Key-DB-Extractor I can successfully extract the whatsapp databases. My whatsapp backups are not password protected.

However, after explorer the wa.db with DB Browser for SqlLite i see that the wa_contacts table remains empty. Other tables like wa_vnames,wa_contact_storage_usage,wa_biz_profiles in this sqlite are populated

Also in Whatsapp on the device all contacts in the Whatsapp conversation have their correct names.

FWIW: When i have to confirm the permission for the backup and press Enter afterwards, i can give the permissions, but i can't open the app because of the Your phone data is inaccurate. Adjust your clock and try again error, - even though the system time is correct - that is caused by the older version of WhatsApp.

Whatsapp Chat Exporter: 

wtsexporter -a

Processing contacts...(0)
Processing messages...(22625/22625)
Processing media...(4410/4410)
Processing vCards...(56/56)
Processing calls...(74)
Generating chats...(376/376)
WhatsApp directory already exists in output directory. Skipping...

Everything is done!

Extracted files:

Name                Length LastWriteTime
----                ------ -------------
.placeholder             2 20:34:43
axolotl.db          299008 22:00:55
chatsettings.db      24576 21:29:43
msgstore.db       18329600 20:34:52
wa.db               417792 20:37:41
wa.db-shm            32768 20:42:08
wa.db-wal                0 20:42:08
whatsapp.cryptkey      158 21:29:41

WhatsAppKeyDBExtract output:

Please connect your Android device with USB Debugging enabled:

* daemon not running. starting it now on port 5037 *
* daemon started successfully *

WhatsApp 2.23.18.79 installed

Found legacy WhatsApp 2.11.431 in local folder

Backing up WhatsApp 2.23.18.79
6745 KB/s (52214668 bytes in 7.558s)
Backup complete

Removing WhatsApp 2.23.18.79 skipping data
Success
Removal complete

Installing legacy WhatsApp 2.11.431
3912 KB/s (18329558 bytes in 4.575s)
Success
Install complete

Paused. Open Whatsapp. Ok the Permissions Request. Press Enter to continue:

Now unlock your device and confirm the backup operation.

Please enter your backup password (leave blank for none) and press Enter:

apps/com.whatsapp/f/key
C:\Temp\WhatsApp\WhatsApp-Key-DB-Extractor\bin\tar.exe: apps/com.whatsapp/f/encrypted_backup.key: Not found in archive
C:\Temp\WhatsApp\WhatsApp-Key-DB-Extractor\bin\tar.exe: Error exit delayed from previous errors
apps/com.whatsapp/db/msgstore.db
apps/com.whatsapp/db/wa.db
apps/com.whatsapp/db/axolotl.db
apps/com.whatsapp/db/chatsettings.db
C:\Temp\WhatsApp\WhatsApp-Key-DB-Extractor\bin\tar.exe: apps/com.whatsapp/Avatars: Not found in archive
C:\Temp\WhatsApp\WhatsApp-Key-DB-Extractor\bin\tar.exe: Error exit delayed from previous errors

Extracting whatsapp.cryptkey ...
Extracting msgstore.db ...
Extracting wa.db ...
Extracting axolotl.db ...
Extracting chatsettings.db ...

Pushing cipher key to: /sdcard/WhatsApp/Databases/.nomedia
5 KB/s (158 bytes in 0.027s)

Cleaning up temporary files ...
Done

Restoring WhatsApp 2.23.18.79
4923 KB/s (52214668 bytes in 10.355s)
Success
Restore complete

Removing WhatsApp 2.23.18.79 temporary apk

Operation complete
KnugiHK commented 9 months ago

Hi. Would you be able to find where the contact names came from?

KnugiHK commented 9 months ago

When i have to confirm the permission for the backup and press Enter afterwards, i can give the permissions, but i can't open the app because of the Your phone data is inaccurate. Adjust your clock and try again error, - even though the system time is correct - that is caused by the older version of WhatsApp.

This is normal.

However, after explorer the wa.db with DB Browser for SqlLite i see that the wa_contacts table remains empty. Other tables like wa_vnames,wa_contact_storage_usage,wa_biz_profiles in this sqlite are populated

Also in Whatsapp on the device all contacts in the Whatsapp conversation have their correct names.

I can reproduce the problem. But further investigations are needed.

sietsevdschoot commented 9 months ago

Hi. Would you be able to find where the contact names came from?

The wa.db contains a table system_contacts_version_table with a id and version column. In my database this table contains about 2000 rows. It might have something to do with it.

The contacts app on my Android, synced with Gmail contacts, contains all contacts used in the WhatsApp conversations.

sietsevdschoot commented 9 months ago

When i have to confirm the permission for the backup and press Enter afterwards, i can give the permissions, but i can't open the app because of the Your phone data is inaccurate. Adjust your clock and try again error, - even though the system time is correct - that is caused by the older version of WhatsApp.

This is normal.

However, after explorer the wa.db with DB Browser for SqlLite i see that the wa_contacts table remains empty. Other tables like wa_vnames,wa_contact_storage_usage,wa_biz_profiles in this sqlite are populated Also in Whatsapp on the device all contacts in the Whatsapp conversation have their correct names.

I can reproduce the problem. But further investigations are needed.

I've tried several ways of trying to force whatsapp to refresh the contact list. But none of those options caused the contacts table to get populated.

So i took another approach. I downloaded all contacts from Gmail in .vcf format. I've found a nice Powershell script to convert .vcf on Github ConvertFrom-Vcf. I had to tweak it a little bit to work for the Gmail format.

After that I wrote a Powershell script to sanitize the phonenumbers and replace all occurences with the actual contactname in the generated .html files, and to rename the file to the contactname.

KnugiHK commented 9 months ago

No sure if WhatsApp changed the database structure again. Any more people encounter this issue?

sietsevdschoot commented 9 months ago

In the meantime I have a script (Github gist) to fix the problem using exported Gmail .vcf contacts to replace contactnames after running the Whatsapp Chat Exporter.

KnugiHK commented 7 months ago

Weird. I just did a new export from WhatsApp 2.23.25.3. The wa_contacts table was populated.

msloco commented 4 months ago

I can confirm same behavior reported by the op happens here with a decrypted wa.db.crypt15 : only populated tables seems to be wa_trusted_contacts and wa_trusted_contacts_send. whatsapp reports version 2.24.3.81.

guilherme-puida commented 4 months ago

The issue also happens here with an export from version 2.24.6.10.

iher1 commented 2 months ago

Hi, I also see the issue on WhatsApp 2.24.8.85 on Android: the wa.db is not populated with any contact data.

qosch commented 1 month ago

Using a WhatsApp database I exported a few years ago (mid 2021), I am experiencing the same issue. Looking at the wa.db file, I can't find any table indicating actual contact names. wa_contact_storage_usage does contain a plausible number of lines, though (but no names).