WhatsApp Parsing Error

[joaovitorlopes]

Recently, when trying to index a UFDR file, in which the WhatsApp application was parsed, with the database and all available attachments, an error occurred when performing this parse on IPED Application. Both versions 4.1.5 and 4.2.0-snapshot were used, and the other applications perform the parser correctly, except WhatsApp. IPED-2024-03-12-08-13-43(4.1.5).log IPED-2024-03-11-16-09-25(4.2.0).log

[lfcnassif]

In the logs I can't see any occurrence or parsing error from msgstore or ChatStorage files, so I think the WhatsApp databases were not exported to the UFDR report.

Please open the UFDR file with 7zip program and try to find files named msgstore.db or ChatStorage.sqlite. If you find them, please share them privately so we can try to reproduce. If you don't find them, that's the expected behavior.

[lfcnassif]

Also see this comment: https://github.com/sepinf-inc/IPED/issues/309#issuecomment-721463990

And the recommendations here: https://github.com/sepinf-inc/IPED/discussions/1541

[joaovitorlopes]

Also see this comment: #309 (comment)

And the recommendations here: #1541

When checking the UFDR file in 7zip it was not possible to find the Databases folder, but when checking in Cellebrite Reader, msgstore.db is there. Regarding the second recommendation, the IPED was indexed directly in the .ufdr and not in the folder where it was located.

[lfcnassif]

When checking the UFDR file in 7zip it was not possible to find the Databases folder

Then possibly it wasn't exported.

but when checking in Cellebrite Reader, msgstore.db is there.

Possibly it is just a folder representing the original file, not the DB itself. Are you able to export the original DB from Cellebrite Reader?

Regarding the second recommendation, the IPED was indexed directly in the .ufdr and not in the folder where it was located.

I referred to all recommendations in that thread given by me and @wladimirleite, not to the user specific issue. For example, I pointed the possible folder (not file) presence theory.

[joaovitorlopes]

Then possibly it wasn't exported.

When analyzing it, it really seems that whoever generated the report ended up removing the databases from the report, and the conversations ended up only being in the ufdr file, thank you in advance for the clarification.

Possibly it is just a folder representing the original file, not the DB itself. Are you able to export the original DB from Cellebrite Reader?

When checking the results in Cellebrite Reader, msgstore.db is actually a folder representing the DB file.

I referred to all recommendations in that thread given by me and @wladimirleite, not to the user specific issue. For example, I pointed the possible folder (not file) presence theory.

Thank you, it is very important to know the hash, it needs to be sha256. For the WhatsApp files link to work.

[wladimirleite]

As far as I remember, this way "msgstore.db" is shown (with 44626 files inside) means it is a folder, not the file, which contains files expanded from the database (like message attachments). It seems the database itself was not included (probably "Databases" category was no selected when generating the UFDR).

PS: @joaovitorlopes, I see you came to a similar conclusion while I was typing.

[joaovitorlopes]

As far as I remember, this way "msgstore.db" is shown (with 44626 files inside) means it is a folder, not the file, which contains files expanded from the database (like message attachments). It seems the database itself was not included (probably "Databases" category was no selected when generating the UFDR).

PS: @joaovitorlopes, I see you came to a similar conclusion while I was typing.

Thank you very much @wladimirleite, it seems that we didn't really export the databases when generating the report.

[lfcnassif]

Thanks @joaovitorlopes for double checking and for returning with the final conclusion.

Anyway, as recommended by the posts I mentioned, you can go to conf/ParsingTaskConfig.txt and change phoneParsersToUse option from internal to external, that will tell IPED to import the WhatsApp decoding results from Cellebrite software instead of decoding the database using IPED internal decoder. external gives different results, better in some aspects: it recovers Android/iOS deleted chats/messages from the main DB itself, recovers scrambled messages from the DB index, may decode payment transactions (not sure), etc. But worse in other aspects: internal links attachments much better because it uses hashes (finding moved, renamed and recovered files), merge Android backups in a single report (recovering deleted messages from backups), recover deleted chats/messages from iOS DBs, is able to download deleted attachments still present in WhatsApp servers (must enable the --downloadInternetData cmd line option), etc.

[lfcnassif]

Anyway, as recommended by the posts I mentioned, you can go to conf/ParsingTaskConfig.txt and change phoneParsersToUse option from internal to external, that will tell IPED to import the WhatsApp decoding results from Cellebrite software instead of decoding the database using IPED internal decoder. external gives different results, better in some aspects: it recovers Android/iOS deleted chats/messages from the main DB itself, recovers scrambled messages from the DB index, may decode payment transactions (not sure), etc. But worse in other aspects: internal links attachments much better because it uses hashes (finding moved, renamed and recovered files), merge Android backups in a single report (recovering deleted messages from backups), recover deleted chats/messages from iOS DBs, is able to download deleted attachments still present in WhatsApp servers (must enable the --downloadInternetData cmd line option), etc.

Just updated this message as a reference to others.