search

KnugiHK / WhatsApp-Chat-Exporter

A customizable Android and iOS/iPadOS WhatsApp database parser that will give you the history of your WhatsApp conversations in HTML and JSON. Android Backup Crypt12, Crypt14, Crypt15, and new schema supported.
https://wts.knugi.dev/
MIT License
584 stars 85 forks source link

Reactions Extraction from SQL Databas #79

Open tlcameron3 opened 8 months ago

tlcameron3 commented 8 months ago

Original Description: I went ahead and grabbed the Reactions data for each message, added the respective lines to the query, and added that data onto the dictionary object in extract.py. Did not interface this updated dictionary object with the HTML output at all. Tested with JSON extraction, worked just fine.

Reactions are understood in the SQL database in two ways: With an associated sender_row_jid: this will be seen in the message_add_on table, and ties relevant information as who performed the reaction. I wanted sender information associated with each reaction, so this worked just fine. The message_add_on also references the message line id, which is what I used to join to the main table. Meanwhile, the message_add_on_reaction simply contains the reaction (an emoji) and the timestamp, as well as a reference to the message_add_on table with its primary key. In this case, only reactions by other users in group chats are accounted for.

The second way the reaction sender is understood is through a "-1" field for the jid. This indicates a reaction associated with the WhatsApp account holder: either a reaction FROM the user to ANY other message (dm or group message), or a reaction to a message in dms with the user (so, the other dm participant reacting to the user's message). This meant doing additional logic in the information that I extracted using group_concatenate in the SQL query, and parsing the concatenated string accordingly in the associated logic. This also means both cases had to have different methods of extracting the reaction sender tag.

Let me know if you would want additional changes to be made in order for this to be merged with main. In my communication on WhatsApp, it became apparent that reactions (usually thumbs-up or hearts) would take the place of a confirmation message (where "okay" or "sounds good" usually sufficed), meaning some conversational information was lost when not taking reactions into account.

The changes account for three additional query lines, and one if-statement of logic, as well as adding "reactions" to the base class attributes. I would be welcome to assist in any other changes to make this branch compatible with your project.

P.S. I fixed the pull request to only include relevant files, previously it included my result files from testing the extraction.

Best, tlc

KnugiHK commented 8 months ago

Thanks for your contribution again. I contacted GitHub support and they removed the previous PR. I will look into your commits later.

mattattackDM commented 6 months ago

Any idea how emoji reactions are stored in the ios db? I've been searching for a while, but can't seem to find it.

mattattackDM commented 6 months ago

Any idea how emoji reactions are stored in the ios db? I've been searching for a while, but can't seem to find it.

Never mind, I found it. Reactions are stored as a blob inside the Zwamessageinfo table.

rounakdatta commented 3 months ago

Can we proceed with this PR?

KnugiHK commented 3 months ago

Can we proceed with this PR?

Unfortunately, it requires additional changes and further review to ensure reliable extraction of reactions.

Even if the PR is merged, reactions won't appear in the outputs because this PR only handles the extraction of reactions from database.