search

KnugiHK / WhatsApp-Chat-Exporter

A customizable Android and iOS/iPadOS WhatsApp database parser that will give you the history of your WhatsApp conversations in HTML and JSON. Android Backup Crypt12, Crypt14, Crypt15, and new schema supported.
https://wts.knugi.dev/
MIT License
518 stars 76 forks source link

The media is missing (even with the media folder available) #81

Closed NoLooseEnds closed 5 months ago

NoLooseEnds commented 5 months ago

I'm having issues with The media is missing, but the chat/text exports fine.

I have the "Media" folder (copied from the phone – and placed in the WhatsApp folder in the working directory. Verified manually that the files are there). I even tried to place it manually with the -m flag.

So not sure what's going on?

Thanks!

BLACK4585 commented 5 months ago

Hey, same issue here. Is it right to copy the WhatsApp folder from /Android/media/com.whatsapp to the root directory of the program?

KnugiHK commented 5 months ago

I'm having issues with The media is missing, but the chat/text exports fine.

I have the "Media" folder (copied from the phone – and placed in the WhatsApp folder in the working directory. Verified manually that the files are there). I even tried to place it manually with the -m flag.

@NoLooseEnds Does #68 helps?

Hey, same issue here. Is it right to copy the WhatsApp folder from /Android/media/com.whatsapp to the root directory of the program?

@BLACK4585 The WhatsApp media folder should be located in the root of your shared storage, i.e. /WhatsApp (or /sdcard/WhatsApp if you know what I am referring to)

BLACK4585 commented 5 months ago

Do you mean the folder in MainStorage/Android/media/com.whatsapp/WhatsApp?

KnugiHK commented 5 months ago

Do you mean the folder in MainStorage/Android/media/com.whatsapp/WhatsApp?

Are your media stored in MainStorage/Android/media/com.whatsapp/WhatsApp/Media?

BLACK4585 commented 5 months ago

Do you mean the folder in MainStorage/Android/media/com.whatsapp/WhatsApp?

Are your media stored in MainStorage/Android/media/com.whatsapp/WhatsApp/Media?

I'm not sure but since I don't have an SD Card inserted.. The folder looks like this: Screenshot_20240131-173227_FX.png

KnugiHK commented 5 months ago

Do you mean the folder in MainStorage/Android/media/com.whatsapp/WhatsApp?

Are your media stored in MainStorage/Android/media/com.whatsapp/WhatsApp/Media?

I'm not sure but since I don't have an SD Card inserted.. The folder looks like this: !

Right. Seems legit. You should copy the whole directory MainStorage/Android/media/com.whatsapp/WhatsApp to the working directory, so that your working directory has a folder called "WhatsApp".

NoLooseEnds commented 5 months ago

@KnugiHK #68 Not really, unless I misunderstand something. I do have a WhatsApp directory in the working directory, and the WhatsApp directory has a Media directory.

Please see tree of working directory (only went down 4 levels – and removed "the middle" where there is a lot of files) and you can tell me if I'm placing it correct?

$ tree -L 4 -I result
├── WhatsApp
│   ├── Backups
│   │   ├── backup_settings.json.crypt14
│   │   ├── chatsettingsbackup.db.crypt14
│   │   ├── commerce_backup.db.crypt14
│   │   ├── statusranking.db.crypt1
│   │   ├── stickers.db.crypt14
│   │   ├── wa.db.crypt14
│   │   └── wallpapers.backup.crypt13
│   ├── Databases
│   │   ├── msgstore-2024-01-14.1.db.crypt14
│   │   ├── msgstore-2024-01-15.1.db.crypt14
│   │   ├── msgstore-2024-01-16.1.db.crypt14
│   │   ├── msgstore-2024-01-17.1.db.crypt14
│   │   ├── msgstore-2024-01-18.1.db.crypt14
│   │   ├── msgstore-2024-01-19.1.db.crypt14
│   │   ├── msgstore-2024-01-20.1.db.crypt14
│   │   ├── msgstore-2024-01-21.1.db.crypt14
│   │   └── msgstore.db.crypt14
│   ├── Media
│   │   ├── WallPaper
│   │   ├── WhatsApp Animated Gifs
│   │   │   ├── Private
│   │   │   └── Sent
│   │   ├── WhatsApp Audio
│   │   │   ├── Private
│   │   │   └── Sent
│   │   ├── WhatsApp Calls
│   │   ├── WhatsApp Documents
│   │   │   ├──  Utlandsbetalning.pdf
│   │   │   ├── DB_Ticket_884594179938.pdf
│   │   │   ├── Private
│   │   │   ├── Sent
│   │   │   └── Tentamen_Stadler.pdf
│   │   ├── WhatsApp Images
│   │   │   ├── IMG-20221030-WA0008.jpg
│   │   │   ├── IMG-20221030-WA0009.jpg
│   │   │   ├── │(…)
│   │   │   ├── IMG-20240121-WA0000.jpg
│   │   │   ├── IMG-20240121-WA0001.jpg
│   │   │   ├── IMG_20230325_145907.jpg
│   │   │   ├── Private
│   │   │   └── Sent
│   │   ├── WhatsApp Profile Photos
│   │   ├── WhatsApp Stickers
│   │   ├── WhatsApp Video
│   │   │   ├── Private
│   │   │   ├── Sent
│   │   │   ├── VID-20230114-WA0008.mp4
│   │   │   ├── VID-20230114-WA0010.mp4
│   │   │   ├── (…)
│   │   │   ├── VID-20230821-WA0005.mp4
│   │   │   ├── VID-20230821-WA0006.mp4
│   │   ├── WhatsApp Video Notes
│   │   ├── WhatsApp Voice Notes
│   │   │   ├── 201825
│   │   │   ├── 201827
│   │   │   ├── (…)
│   │   │   ├── 202352
│   │   │   └── 202403
│   │   ├── thumbnails
│   │   │   ├── 02e3541192674ba26a8d3bac310f0e2c52752ed2daeec6c065895ebe2e96c187.png
│   │   │   ├── 05c663045bf27aa9fe80ae0081020609c9219f6b91b2a40b042830c1fa1c0570.png
│   │   │   ├── (…)
│   │   │   ├── e5ec2d1b0429c59cb881f3a8a9b650e245b92326e2a5bc1a81f882fab7a4b635.png
│   │   │   └── f9b8caedd5aab6dd696349cf9ec5e14aeae2138a58c1cfb032c34d3dc60f6487.png
│   │   └── vCards
│   ├── thumbnails
│   │   ├── 02e3541192674ba26a8d3bac310f0e2c52752ed2daeec6c065895ebe2e96c187.png
│   │   ├── 05c663045bf27aa9fe80ae0081020609c9219f6b91b2a40b042830c1fa1c0570.png
│   │   │   (…)
│   │   ├── e2a686c38f67132bfc08df78e7288507df12eba87c1cf474da61fcd661c1d23a.png
│   │   ├── e5ec2d1b0429c59cb881f3a8a9b650e245b92326e2a5bc1a81f882fab7a4b635.png
│   │   └── f9b8caedd5aab6dd696349cf9ec5e14aeae2138a58c1cfb032c34d3dc60f6487.png
│   └── vCards
├── axolotl.db
├── chatsettings.db
├── msgstore.db
├── tree.log
├── wa.db
└── whatsapp.cryptkey
KnugiHK commented 5 months ago

The The media is missing is shown when the program cannot find your media file. 

file_path = f"{media_folder}/{content['file_path']}"
...
if os.path.isfile(file_path):
    ...
else:
    ...
    message.data = "The media is missing"

The media_folder by default is WhatsApp and the content['file_path'] should be the relative path to the file starting from ./Media. You may want to check the media files location against the paths stated in the database.

NoLooseEnds commented 5 months ago

Am I missing something. Is the working directory supposed to be the WhatsApp directory (with the Media directory inside)? Or should the working directory be whatever and have WhatsApp (still with the Media directory inside)?

KnugiHK commented 5 months ago

Am I missing something. Is the working directory supposed to be the WhatsApp directory (with the Media directory inside)? Or should the working directory be whatever and have WhatsApp (still with the Media directory inside)?

The latter one is correct.

NoLooseEnds commented 5 months ago

Yes, and that's what I have?

The path is pretty far down in the filesystem /Volumes/<hdd-name-mount>/<Team> Dropbox/<full name>/<_directory>/Whatsapp-backup/working_wts but should that matter?

KnugiHK commented 5 months ago

Yes, and that's what I have?

The path is pretty far down in the filesystem /Volumes/<hdd-name-mount>/<Team> Dropbox/<full name>/<_directory>/Whatsapp-backup/working_wts but should that matter?

How many characters does this path has? In another issue, long path did affect the program.

NoLooseEnds commented 5 months ago

76 chars, but it also contains space and a nordic char. Anyhow, I just moved it to a shorter non space non nordic char directory /Users/username/Desktop to be precise, and I still experience the same issue. And it seems like it sees the media and copies it, but I still get The media is missing

$ ~/Desktop/working_wts  wtsexporter -a
Processing contacts...(284)
Processing messages...(56623/56623)
Processing media...(9639/9639)
Processing vCards...(4/4)
Processing calls...(664)
Generating chats...(283/283)
Copying media directory...

Everything is done!
KnugiHK commented 5 months ago

76 chars, but it also contains space and a nordic char. Anyhow, I just moved it to a shorter non space non nordic char directory /Users/username/Desktop to be precise, and I still experience the same issue. And it seems like it sees the media and copies it, but I still get The media is missing

$ ~/Desktop/working_wts  wtsexporter -a
Processing contacts...(284)
Processing messages...(56623/56623)
Processing media...(9639/9639)
Processing vCards...(4/4)
Processing calls...(664)
Generating chats...(283/283)
Copying media directory...

Everything is done!

Copying and finding a media file are two different logics and they are not related. I think in this case you should take a look into the database.

NoLooseEnds commented 5 months ago

Ok, not totally sure what to look for, but I found a table called media_refs and the path it uses is: /storage/emulated/0/WhatsApp/Media/WhatsApp Images/<varies>/filename.jpg (it's from an Huawei P30 pro phone)

"/storage/emulated/0/WhatsApp/Media/WhatsApp Images/Sent/IMG-20190720-WA0004.jpg"
"/storage/emulated/0/WhatsApp/Media/WhatsApp Images/Sent/IMG-20190910-WA0013.jpg"
"/storage/emulated/0/WhatsApp/Media/WhatsApp Images/IMG-20190911-WA0015.jpg"
"/storage/emulated/0/WhatsApp/Media/WhatsApp Images/Sent/IMG-20190911-WA0019.jpg"
"/storage/emulated/0/WhatsApp/Media/WhatsApp Images/Sent/IMG-20190907-WA0021.jpg"
"/storage/emulated/0/WhatsApp/Media/WhatsApp Images/Sent/IMG-20191030-WA0001.jpg"
"/storage/emulated/0/WhatsApp/Media/WhatsApp Video/Sent/VID-20190918-WA0007.mp4"
"/storage/emulated/0/WhatsApp/Media/WhatsApp Images/Sent/IMG-20191220-WA0016.jpg"
"/storage/emulated/0/WhatsApp/Media/WhatsApp Images/Sent/IMG-20180611-WA0007.jpg"
"/storage/emulated/0/WhatsApp/Media/WhatsApp Images/Sent/IMG-20180611-WA0005.jpg"
"/storage/emulated/0/WhatsApp/Media/WhatsApp Animated Gifs/Sent/VID-20191018-WA0009.mp4"
"/storage/emulated/0/WhatsApp/Media/WhatsApp Images/Sent/IMG-20200130-WA0000.jpg"
"/storage/emulated/0/WhatsApp/Media/WhatsApp Stickers/STK-20200131-WA0003.webp"
"/storage/emulated/0/WhatsApp/Media/WhatsApp Images/IMG-20190912-WA0003.jpg"
"/storage/emulated/0/WhatsApp/Media/WhatsApp Stickers/STK-20200131-WA0004.webp"
"/storage/emulated/0/WhatsApp/Media/WhatsApp Animated Gifs/VID-20200222-WA0000.mp4"
KnugiHK commented 5 months ago

Ok, not totally sure what to look for, but I found a table called media_refs and the path it uses is: /storage/emulated/0/WhatsApp/Media/WhatsApp Images/Sent/filename.jpg (it's from an Huawei P30 pro phone)

Copy the right query for your database in the extract.py, and look for the file_path column.

NoLooseEnds commented 5 months ago

I think I might have messed up the copy of the WhatsApp folder. On further investigation it looks like I only have newer-ish files – giving me missing media in the beginning of all the chats, but showing media files at the end. So it seems like the issue is with me and not the software (as often is the case it feels like when debugging).

Appreciate the help, but I think this can be closed.

KnugiHK commented 5 months ago

I think I might have messed up the copy of the WhatsApp folder. On further investigation it looks like I only have newer-ish files – giving me missing media in the beginning of all the chats, but showing media files at the end. So it seems like the issue is with me and not the software (as often is the case it feels like when debugging).

Appreciate the help, but I think this can be closed.

I see. The (really) missing media used to concerns me as well. But I rarely use WhatsApp now.

@BLACK4585 Any further question/issue on the matter? If not, I am closing this.

BLACK4585 commented 5 months ago

Oh yeah, same issue for me. The media files at the end are present, seems like the old media files are simply not existent anymore. Thank you very much for your help!