Invalid Start Byte in email

vi-or-die commented 6 years ago

Started to see a decline in the amount of phishing emails being reported and noticed that imap2thehive wasn't importing certain messages.

Digging into my error logs on the script, I noticed that an error is being generated when it attempts to decode the message. Doing some searching on that specific byte 0xA9 is é. However other searches on the exact error message shows that there might be a problem with the way its being decoded.

Error Seen:

Traceback (most recent call last):
  File "/etc/imap2thehive/imap2thehive.py", line 496, in <module>
    main()
  File "/etc/imap2thehive/imap2thehive.py", line 492, in main
    readMail(mailConnect())
  File "/etc/imap2thehive/imap2thehive.py", line 377, in readMail
    if submitTheHive(message) == True:
  File "/etc/imap2thehive/imap2thehive.py", line 211, in submitTheHive
    body = part.get_payload(decode=True).decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa9 in position 16493: invalid start byte

xme commented 6 years ago

I'll have a look. Probably an UTF issue...

xme commented 6 years ago

As a temporary workaround, you can replace line 211 with: body = part.get_payload(decode=True).decode(errors='ignore')

vi-or-die commented 6 years ago

Thanks @xme, that helped!

After adding that on line 211 I had a similar issue for line 216. I did the same thing by modifying the line to html = part.get_payload(decode=True).decode(errors='ignore') this seemed to get things moving again.

Error Recieved

Traceback (most recent call last):
  File "/etc/imap2thehive/imap2thehive.py", line 496, in <module>
    main()
  File "/etc/imap2thehive/imap2thehive.py", line 492, in main
    readMail(mailConnect())
  File "/etc/imap2thehive/imap2thehive.py", line 377, in readMail
    if submitTheHive(message) == True:
  File "/etc/imap2thehive/imap2thehive.py", line 216, in submitTheHive
    html = part.get_payload(decode=True).decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe7 in position 2477: invalid continuation byte

xme / dockers

Invalid Start Byte in email #8

Error Recieved