B16f00t / whapa

WhatsApp Parser Toolset v1.59
1.1k stars 258 forks source link

No parsed messages in whatsapp version 2.22.7.74 #164

Open rafaelcsch opened 2 years ago

rafaelcsch commented 2 years ago

No parsed messages in whatsapp version 2.22.7.74 since there is no more "messages" table.

karanrajpal14 commented 2 years ago

Yeah I've been facing this issue as well. Not sure what's the best way to deal with this.

B16f00t commented 2 years ago

yes, now the information is in the message table, when I have some time I will update it

apremi commented 2 years ago

+1

vb0 commented 2 years ago

+1
This is breaking completely all parsing for data coming from new (and as time goes by not so new) versions of Whatsapp.

apremi commented 2 years ago

I fixed the issue on another parser. I can help. I will try to update the solution here in the evening after work.

nufrankz commented 2 years ago

I fixed the issue on another parser. I can help. I will try to update the solution here in the evening after work.

Thanks, we'll patiently wait for it 😬

B16f00t commented 2 years ago

If you have any updates you can make a pull request.

stevanuscolonne commented 1 year ago

Any update on this?

ghost commented 1 year ago

any update?

ChristianMarchiori commented 1 year ago

this may help https://thebinaryhick.blog/2022/06/09/new-msgstore-who-dis-a-look-at-an-updated-whatsapp-on-android/

ChristianMarchiori commented 1 year ago

I fixed the issue on another parser. I can help. I will try to update the solution here in the evening after work.

On which parser did u fix it?

aadionizio commented 1 year ago

+1 This is breaking completely all parsing for data coming from new (and as time goes by not so new) versions of Whatsapp.

Thanks @vb0 (solution link) With that I managed to get whapa to work, at least partially. It now shows a lot of errors, but at least produces a report. There are some problems with whatsapp media not being linked, but hope this helps. Use a sqlite editor and run this queries on whatsapp database:

DROP TABLE IF EXISTS messages_quotes;
CREATE TABLE IF NOT EXISTS messages_quotes AS 
SELECT message_quoted.message_row_id AS _id,
        message_quoted.chat_row_id,
        message_quoted.parent_message_chat_row_id,
        message_quoted.from_me AS key_from_me,
        message_quoted.sender_jid_row_id,
        message_quoted.key_id,
        message_quoted.timestamp,
        message_quoted.message_type AS media_wa_type,
        message_quoted.text_data AS data,
        message_quoted.payment_transaction_id,
        message_quoted.lookup_tables,
        message_quoted.origin,
        receipts.key_remote_jid,
        message.text_data AS data,
        message.status,
        NULL AS needs_push,
        message_media.message_url AS media_url,
        message_media.mime_type AS media_mime_type,
        message_media.file_size AS media_size,
        message_media.media_name,
        message_media.media_caption,
        message_media.file_hash AS media_hash,
        message_media.media_duration,
        message_location.latitude,
        message_location.longitude,
        message_media.file_path AS thumb_image,
        receipts.remote_resource,
        message.received_timestamp,
        NULL AS send_timestamp,
        message.receipt_server_timestamp,
        receipts.receipt_device_timestamp,
        receipts.read_device_timestamp,
        receipts.played_device_timestamp,
        message_thumbnail.thumbnail AS raw_data,
        message.recipient_count,
        message.participant_hash,
        message.starred,
        message._id AS quoted_row_id,
        NULL AS mentioned_jids,
        NULL AS multicast_id,
        NULL AS edit_version,
        message_media.enc_file_hash AS media_enc_hash,
        message.origination_flags AS forwarded,
        NULL AS preview_type,
        NULL AS send_count,
        message_future.future_message_type,
        message.message_add_on_flags
FROM message_quoted
    LEFT JOIN message ON message_quoted.message_row_id = message._id
    LEFT JOIN receipts ON message.key_id = receipts.key_id
    LEFT JOIN message_media ON message_quoted.message_row_id = message_media.message_row_id
    LEFT JOIN message_location ON message_quoted.message_row_id = message_location.message_row_id
    LEFT JOIN message_thumbnail ON message_quoted.message_row_id = message_thumbnail.message_row_id
    LEFT JOIN message_future ON message_quoted.message_row_id = message_future.message_row_id;

DROP TABLE IF EXISTS messages_links;
CREATE TABLE IF NOT EXISTS messages_links AS
SELECT message_link._id,
        message_link.chat_row_id,
        message_link.message_row_id,
        message_link.link_index,
        receipts.key_remote_jid
FROM message_link
    LEFT JOIN message ON message_link.message_row_id = message._id
    LEFT JOIN receipts ON message.key_id = receipts.key_id;

DROP TABLE IF EXISTS messages;
CREATE TABLE IF NOT EXISTS messages AS
SELECT message._id,
       jid.raw_string  AS key_remote_jid,
       message.from_me AS key_from_me,
       message.key_id,
       message.status,
       NULL AS needs_push,
       message.text_data AS data,
       message.timestamp,
       message_media.message_url AS media_url,
       message_media.mime_type AS media_mime_type,
       message.message_type AS media_wa_type,
       message_media.file_size AS media_size,
       message_media.media_name,
       message_media.media_caption,
       message_media.file_hash AS media_hash,
       message_media.media_duration,
       message.origin,
       message_location.latitude,
       message_location.longitude,
       message_media.file_path AS thumb_image,
       receipts.remote_resource,
       message.received_timestamp,
       NULL AS send_timestamp,
       message.receipt_server_timestamp,
       receipts.receipt_device_timestamp,
       receipts.read_device_timestamp,
       receipts.played_device_timestamp,
       message_thumbnail.thumbnail AS raw_data,
       message.recipient_count,
       message.participant_hash,
       message.starred,
       message._id AS quoted_row_id,
       NULL AS mentioned_jids,
       NULL AS multicast_id,
       NULL AS edit_version,
       message_media.enc_file_hash AS media_enc_hash,
       NULL AS payment_transaction_id,
       message.origination_flags AS forwarded,
       NULL AS preview_type,
       NULL AS send_count,
       message.lookup_tables,
       message_future.future_message_type,
       message.message_add_on_flags
FROM message
        LEFT JOIN message_media ON message._id = message_media.message_row_id
        LEFT JOIN message_location ON message._id = message_location.message_row_id
        LEFT JOIN message_thumbnail ON message._id = message_thumbnail.message_row_id
        LEFT JOIN message_future ON message._id = message_future.message_row_id
        LEFT JOIN receipts ON message.key_id = receipts.key_id
        LEFT JOIN chat ON message.chat_row_id = chat._id
        LEFT JOIN jid ON chat.jid_row_id = jid._id;

EDIT: there's also a problem with group chats

formalist commented 1 year ago
SELECT message_quoted.message_row_id AS _id,
        message_quoted.chat_row_id,
   [...]
        NULL AS edit_version,

The field edit_version was used for deleted messages, to indicate if 'deleted for me' (=5) or 'deleted for all (=7).

Here you are forcing NULL... I know this is a workaround, but does anyone know where this information is located, in the new database schema?

nkovacne commented 1 year ago

Is this going to be solved at some point?