No such table: messages (Edit: Support for WhatsApp databases with "message" table)

[Takepy]

When running the script I get following error message

Gathering contacts...(97)
Traceback (most recent call last):
  File "C:\Users\******\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\******\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\******\AppData\Local\Programs\Python\Python310\Scripts\wtsexporter.exe\__main__.py", line 7, in <module>
  File "C:\Users\******\AppData\Local\Programs\Python\Python310\lib\site-packages\Whatsapp_Chat_Exporter\__main__.py", line 147, in main
    messages(db, data)
  File "C:\Users\******\AppData\Local\Programs\Python\Python310\lib\site-packages\Whatsapp_Chat_Exporter\extract.py", line 88, in messages
    c.execute("""SELECT count() FROM messages""")
sqlite3.OperationalError: no such table: messages

My database doesn't contain the table messages, I only have one called message which contains the data but the schema of the table looks different from what it's supposed to look like. What could be the issue here?

[KnugiHK]

Hi, sorry for the delay; I didn't have the old backup available immediately.

1. Nope, the "This message is not supported" thing does not display in the resulting HTML when converting an old database. The chat is actually complete, so maybe it is the "message" saying: "Messages and calls are end-to-end encrypted..."

2. The 'main' branch doesn't have the '-s' flag available at all.

Metadata is also treated as messages in the database. But since it is not the main focus when it comes to exporting chats, metadata may show as "not supported" unless handled like changes of the group name. Therefore, I will consider it not a bug if the chat is complete.

I will start preparing for the release when more tests are being performed.

[lordfeck]

I checked out the message_table branch and used it to extract my newer 'message' DB successfully. All conversations including group conversations were converted to HTML. Many thanks @KnugiHK !

[nahoj]

I had to cherry-pick the fix for https://github.com/KnugiHK/Whatsapp-Chat-Exporter/issues/32 and then it worked perfectly. Thank you for making this tool!

➜  Whatsapp-Chat-Exporter git:(message_table)$ git cherry-pick 2ba5571
➜  Whatsapp-Chat-Exporter git:(message_table)$ pip install '.[android_backup]'

[KnugiHK]

Hello everyone! We are finally moving to the last step before releasing this update.

I merged the logic for processing old and new schema in extract.py in commit fb88124. As it is not a commit specifically for the message table, it will not be merged into the message_table branch, and the branch will not be updated from now on. And changes will be committed to the dev branch as usual.

Feel free to try it, especially for those with databases with both message and messages tables, to see if there is any compatibility issue.

[umjediqualquer]

Hey there! Not sure if it's related, but I was having the same issue related to this thread, downloaded both the commit and the dev branch versions and now I'm getting a different error:

Traceback (most recent call last):
  File "/home/user/.local/bin/wtsexporter", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/Whatsapp_Chat_Exporter/__main__.py", line 229, in main
    messages(db, data)
  File "/home/user/.local/lib/python3.11/site-packages/Whatsapp_Chat_Exporter/extract.py", line 249, in messages
    data[content["key_remote_jid"]].add_message(content["_id"], Message(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'dict' object has no attribute 'add_message'

Again, not sure if it's related, but what should I do in this case, please?

[KnugiHK]

Hey there! Not sure if it's related, but I was having the same issue related to this thread, downloaded both the commit and the dev branch versions and now I'm getting a different error:

Traceback (most recent call last):
  File "/home/user/.local/bin/wtsexporter", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/Whatsapp_Chat_Exporter/__main__.py", line 229, in main
    messages(db, data)
  File "/home/user/.local/lib/python3.11/site-packages/Whatsapp_Chat_Exporter/extract.py", line 249, in messages
    data[content["key_remote_jid"]].add_message(content["_id"], Message(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'dict' object has no attribute 'add_message'

Again, not sure if it's related, but what should I do in this case, please?

@umjediqualquer What's the command you issued and does wa.db exists?

[umjediqualquer]

Hiya there @KnugiHK, I ran wtsexporter -a. I have msgstore.db, wa.db and the WhatsApp folder (containing the Backups, Databases and Media folders) in the directory I tried to run the command.

[KnugiHK]

Hiya there @KnugiHK, I ran wtsexporter -a. I have msgstore.db, wa.db and the WhatsApp folder (containing the Backups, Databases and Media folders) in the directory I tried to run the command.

@umjediqualquer I did a quick fix on it (b371587) in the dev branch. Would you please check if the fix works?

[umjediqualquer]

@umjediqualquer I did a quick fix on it (b371587) in the dev branch. Would you please check if the fix works?

Managed to make some progress, getting a different error message now:

Gathering messages...(694764/694764)
Gathering media...(43618/43618)
Traceback (most recent call last):
  File "/home/user/.local/bin/wtsexporter", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/Whatsapp_Chat_Exporter/__main__.py", line 231, in main
    vcard(db, data)
  File "/home/user/.local/lib/python3.11/site-packages/Whatsapp_Chat_Exporter/extract.py", line 491, in vcard
    file_name = "".join(x for x in media_name if x.isalnum())
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not iterable```

[KnugiHK]

@umjediqualquer I did a quick fix on it (b371587) in the dev branch. Would you please check if the fix works?

Managed to make some progress, getting a different error message now:

Gathering messages...(694764/694764)
Gathering media...(43618/43618)
Traceback (most recent call last):
  File "/home/user/.local/bin/wtsexporter", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/Whatsapp_Chat_Exporter/__main__.py", line 231, in main
    vcard(db, data)
  File "/home/user/.local/lib/python3.11/site-packages/Whatsapp_Chat_Exporter/extract.py", line 491, in vcard
    file_name = "".join(x for x in media_name if x.isalnum())
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not iterable```

Do you know if your msgstore.db use message or messages table?

[umjediqualquer]

Do you know if your msgstore.db use message or messages table?

I'm assuming it's message because when running the release version, I get the sqlite3.OperationalError: no such table: messages error, but I can open this with a db viewer to double check.

[KnugiHK]

Do you know if your msgstore.db use message or messages table?

I'm assuming it's message because when running the release version, I get the sqlite3.OperationalError: no such table: messages error, but I can open this with a db viewer to double check.

Attempt on fixing it is pushed (06a1d34). See if that works (the error may disappear but it is not a good fix IMO).

[umjediqualquer]

Attempt on fixing it is pushed (06a1d34). See if that works (the error may disappear but it is not a good fix IMO).

The previous error has been fixed, but it threw a new error a few lines after:

Gathering messages...(694764/694764)
Gathering media...(43618/43618)
Traceback (most recent call last):
  File "/home/user/.local/bin/wtsexporter", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/Whatsapp_Chat_Exporter/__main__.py", line 231, in main
    vcard(db, data)
  File "/home/user/.local/lib/python3.11/site-packages/Whatsapp_Chat_Exporter/extract.py", line 496, in vcard
    data[row["key_remote_jid"]].messages[row["message_row_id"]].data = media_name + \
                                                                       ^^^^^^^^^^^^^^
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

Is there a way to extract only the chat without the media? It's weekend and I don't to keep bothering you with this haha

[KnugiHK]

Is there a way to extract only the chat without the media?

There is no way to extract chat only. Maybe it is a good option to add.

It's weekend and I don't to keep bothering you with this haha

That's fine. The bug has to be fixed anyway. And I reply only when I am available.

As for the bug, I committed another attempt to fix it (7c0b90d). This time I modified my database to test the problem myself and it works for me. Also, maybe the file name should rely on the information inside the vcards instead of the message data (the current implementation).

[umjediqualquer]

@KnugiHK it worked! Seems to have generated everything correctly and didn't throw any errors :D

Thank you very much for your help, especially on a weekend! May I buy you a cup of coffee (or tea, it that's your cup, heh) as a token of gratitude?

[KnugiHK]

@KnugiHK it worked! Seems to have generated everything correctly and didn't throw any errors :D

Thank you very much for your help, especially on a weekend! May I buy you a cup of coffee (or tea, it that's your cup, heh) as a token of gratitude?

Glad that works for you too! And thanks for your report! The point for not releasing this update yet is to let everyone test it and spot problems.

Also, I appreciate your offer to buy me a cup of coffee or tea, but it's not necessary. If you'd like to show your support, consider staring this repo instead. That meant a lot to me😆. Thanks again.

[Signum21]

Hi, I cloned the "messagetable" branch and run the tool with this command: python3 __main_\.py -a --db ./msgstore.db

The messages are recovered correctly but i noticed that two identical "WhatsApp" folders are created. One outside and one inside the "result" folder. This might be a bug.

The two folders structure are:

./WhatsApp/ ./WhatsApp/vCards/

./result/ ./result/messages.html ./result/WhatsApp/ ./result/WhatsApp/vCards/

[KnugiHK]

Hi, I cloned the "message_table" branch and run the tool with this command: python3 main.py -a --db ./msgstore.db

The messages are recovered correctly but i noticed that two identical "WhatsApp" folders are created. One outside and one inside the "result" folder. This might be a bug.

The two folders structure are:

./WhatsApp/ ./WhatsApp/vCards/

./result/ ./result/messages.html ./result/WhatsApp/ ./result/WhatsApp/vCards/

@Signum21 Did "Copying media directory..." printed in your terminal? I guess your problem is related to #25.

Also, "message_table" branch is no longer maintained. Try dev branch instead.

[Signum21]

I switched to dev branch. "Copying media directory..." printed in the terminal. I think the problem is not related to #25 cause I don't have the media folder so it's not a problem of moving instead of copying, it's creating two copies of the same folder from zero. I'm working only with the msgstore.db file.

[KnugiHK]

I switched to dev branch. "Copying media directory..." printed in the terminal. I think the problem is not related to #25 cause I don't have the media folder so it's not a problem of moving instead of copying, it's creating two copies of the same folder from zero. I'm working only with the msgstore.db file.

vCards will be exported to ./WhatsApp/vCards and copied to the output directory by default. If you don't want two identical folders, try -c (--move-media) flag.

[KnugiHK]

The output treats deleted message as not supported message in the new schema. It is fixed in c7a01bb.

[UsamaAshfaq]

WhatsApp using "message" table spread different types of message to many other tables. Good chance for me to rewrite the project :)

Kindly do it, I am also getting same error : sqlite3.OperationalError: no such table: messages In my case Messages are stored in "message" table.

[KnugiHK]

WhatsApp using "message" table spread different types of message to many other tables. Good chance for me to rewrite the project :)

Kindly do it, I am also getting same error : sqlite3.OperationalError: no such table: messages In my case Messages are stored in "message" table.

The support of the message table is completed and will be released soon.

[KnugiHK]

Support with new schema has been released in 0.9.0. I am closing this issue. Further problems should be reported on a dedicated issue. Thanks to everyone who contributed to the support of the new schema!