KnugiHK / WhatsApp-Chat-Exporter

A customizable Android and iOS/iPadOS WhatsApp database parser that will give you the history of your WhatsApp conversations in HTML and JSON. Android Backup Crypt12, Crypt14, Crypt15, and new schema supported.
https://wts.knugi.dev/
MIT License
521 stars 76 forks source link

OSError: [Errno 22] Invalid argument (Negative timestamp on iOS) #19

Open mhsshark opened 1 year ago

mhsshark commented 1 year ago

I have the below error while extracting data for an iPhone backup:

File "C:\Users\[user]\AppData\Local\Programs\Python\Python39\lib\site-packages\Whatsapp_Chat_Exporter\extract_iphone.py", line 69, in messages
    "time": datetime.fromtimestamp(ts).strftime("%H:%M"),
OSError: [Errno 22] Invalid argument

To make it clear, some media files were extracted already.

KnugiHK commented 1 year ago

Are you able to print out these two variables, APPLE_TIME and content[3] in line 65 of extract_iphone.py?

mhsshark commented 1 year ago

Are you able to print out these two variables, APPLE_TIME and content[3] in line 65 of extract_iphone.py?

Generally I have no experience with python, however I'm sure I will be able to do it if you guide me

KnugiHK commented 1 year ago
  1. Open the python file (C:\Users[user]\AppData\Local\Programs\Python\Python39\lib\site-packages\Whatsapp_Chat_Exporter\extract_iphone.py) with text editor.
  2. If you are using the latest version, add a print line between line 64 and 65 as follow.
    while content is not None:
        if content[0] not in data:
            data[content[0]] = {"name": None, "messages": {}}
        print(f"{APPLE_TIME=}, {content[3]=}")
        ts = APPLE_TIME + content[3]
  3. Run the program. Before the error stack trace pops up, there should be a line showing the variables just like the following code block. Redact any sensitive information and post it here along with the stack trace.
    APPLE_TIME=123456789, content[3]=987
mhsshark commented 1 year ago

See the below results:

APPLE_TIME=978296400.0, content[3]=633263721
APPLE_TIME=978296400.0, content[3]=-978307423.877771
Traceback (most recent call last):
  File "C:\Users\[user]\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\[user]\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\[user]\AppData\Local\Programs\Python\Python39\Scripts\wtsexporter.exe\__main__.py", line 7, in <module>
  File "C:\Users\[user]\AppData\Local\Programs\Python\Python39\lib\site-packages\Whatsapp_Chat_Exporter\__main__.py", line 174, in main
    messages(db, data)
  File "C:\Users\[user]\AppData\Local\Programs\Python\Python39\lib\site-packages\Whatsapp_Chat_Exporter\extract_iphone.py", line 70, in messages
    "time": datetime.fromtimestamp(ts).strftime("%H:%M"),
OSError: [Errno 22] Invalid

I can see the content[3] = negative value for whatever reason.. let me know if I need to do something else to help

KnugiHK commented 1 year ago

First, we need to determine which message the timestamp belongs to by looking at the WhatsApp database extracted. Then, identify the human-readable time of the message through the WhatsApp app. Doing so may allow us to understand why negative value is presented.

kabutor commented 1 year ago

I'm`` having the same error i did a print(content) :

`APPLE_TIME=978303600.0, content[3]=-980367545.6843619

('3XXredactedXXX431@s.whatsapp.net', 84232, 0, -980367545.6843619, None, 10, None) Gathering messages...(35999/35999) `

I did a dirty fix/hack :(

if (content[3] < 0):
    print(f"{APPLE_TIME=}, {content[3]=}")
    print(content)
    ts = APPLE_TIME
else:
    ts = APPLE_TIME + content[3]

I like to put this chat into an android backup to restore it on an android phone but I guess is not possible

KnugiHK commented 1 year ago

I'm`` having the same error i did a print(content) :

`APPLE_TIME=978303600.0, content[3]=-980367545.6843619

('3XXredactedXXX431@s.whatsapp.net', 84232, 0, -980367545.6843619, None, 10, None) Gathering messages...(35999/35999) `

I did a dirty fix/hack :(

if (content[3] < 0):
    print(f"{APPLE_TIME=}, {content[3]=}")
    print(content)
    ts = APPLE_TIME
else:
    ts = APPLE_TIME + content[3]

I like to put this chat into an android backup to restore it on an android phone but I guess is not possible

When is your iPhone backup be created?

kabutor commented 1 year ago

I did the backup the same day I posted the message, if you want me to try some things, I still have the backup and can do any change you want me to do

KnugiHK commented 1 year ago

We will need the human-readable time and its corresponding raw timestamp to understand how should we interpret the timestamp. You may do so by following these steps:

  1. Checkout to dev branch.
  2. Show the raw timestamp in the HTML by replacing time=ts, with time=str(content["ZMESSAGEDATE"]),
        if (content["ZMESSAGEDATE"] < 0):
            ts = APPLE_TIME
        else:
            ts = APPLE_TIME + content["ZMESSAGEDATE"] # keep your hack here for now
        message = Message(
            from_me=content["ZISFROMME"],
            timestamp=ts,
            time=str(content["ZMESSAGEDATE"]),
            key_id=content["ZSTANZAID"][:17],
        )
  3. Select one of the message in the output HTML files and find that message in WhatsApp.
  4. Paste the raw timestamp printed in the HTML and the human-readable time in WhatsApp of the selected message here.
  5. Repeat the step 3 and 4 for another message that is close to last one. Note that they should have an interval for at least 2 minutes.
kabutor commented 1 year ago

well, I have a problem with that, I can't open whatsapp on the device, so I can't check the "Human-readable" time of the message.

I'll try to report more info from the messages that fail to see if that helps (I think are only 2 entries of the 35999)

KnugiHK commented 1 year ago

If someone has the same problem, feel free to reproduce the steps I mentioned and report back. Meanwhile, any thoughts on the negative timestamp are welcome.

kabutor commented 1 year ago

I did look for the two messages that have the negative input in the exported html logs.

My guess is that those are deleted messages, but as I can't see the whatsapp app is only a guess, find enclosed the pictures (the messages are inserted in the what looks like the right position but with the 2001-01-01 date)

capt1 capt2

KnugiHK commented 1 year ago

I did look for the two messages that have the negative input in the exported html logs.

My guess is that those are deleted messages, but as I can't see the whatsapp app is only a guess, find enclosed the pictures (the messages are inserted in the what looks like the right position but with the 2001-01-01 date)

capt1 capt2

@kabutor Can you try to rerun the tool with the dev branch? Any deleted messages should now be shown as "deleted message" correctly instead of displaying "This message is not supported". Doing so we can be sure if it is related to deleted message.

kabutor commented 1 year ago

I'm sorry I don't have those files anymore.

A woman came to me, their husband and daughter has passed away in a car accident, I just help her to retrieve the whatsapps between her and her deceased relatives in a broken phone, as she find comfort on reading the conversations she had with them when they were alive, and your script was very helpful.

I thought you may find interesting that the software you created helped someone in a deep way. Thanks

KnugiHK commented 1 year ago

I'm sorry I don't have those files anymore.

A woman came to me, their husband and daughter has passed away in a car accident, I just help her to retrieve the whatsapps between her and her deceased relatives in a broken phone, as she find comfort on reading the conversations she had with them when they were alive, and your script was very helpful.

I thought you may find interesting that the software you created helped someone in a deep way. Thanks

Sorry to hear that. Indeed, I'm grateful that the exporter was able to provide some comfort and support to someone in archiving the WhatsApp conversations.