search

bbottema / outlook-message-parser

A Java parser for Outlook messages (.msg files)
76 stars 35 forks source link

Handling signed messages (smime.p7m attachments) #4

Closed gasparez15 closed 5 years ago

gasparez15 commented 5 years ago

Hi Benny, I wanted to ask you if this library manages the signed Outlook message files. Using the msgparser library I see that this message has an IPM.Note.SMIME class and that there is only one attachment called smime.p7m This message has no body, but only an attachment Is this library able to interpret the body and the attachments contained in the attachment smime? Outlook Signed Message.zip Thanks 4 Y time Alex

bbottema commented 5 years ago

No, I'm not familiar with this and the underlying library used doesn't seem to do anything with it. I'll see what I can find out about it.

bbottema commented 5 years ago

I've had success in reading this smime attachment (using java-utils-mail-smime) and indeed found the content which is just an .eml, but I'm unsure how to handle this scenario properly.

If I open your supplied .eml in Thunderbird and that .msg in Outlook, both clients treat the attachment as the actual message. Is this behavior specified somewhere?

Currently I would be able to return the empty message with an .eml file attachment. This EML can easily be converted then to an Email object from Simple Java Mail, which provides a clean API for further content extraction.

OutlookFileAttachment eml = msg.fetchTrueAttachments().get(0);
MimeMessage mimeMessage = new MimeMessage(null, new ByteArrayInputStream(eml.getData()));

// Simple Java Mail Magic:
Email email = EmailConverter.mimeMessageToEmail(mimeMessage);

If this behavior would meet general expectation, I will go ahead with this solution.

bbottema commented 5 years ago

In fact, since this is not actually Outlook specific (signed attachments), I'm going to move S/MIME support over to Simple Java Mail. That's because to support your particular case (signed MimeMessage), outlook-message-parser would have to add javax.mail and bouncy castle as new dependencies, which are rather heavy weight.

Simple Java Mail already has javax.mail and works with optional dependencies, of which bouncy castle can be one. More to the point, I would like Simple Java Mail to allow new mails to be signed as well.

gasparez15 commented 5 years ago

Ok Benny, in fact I agree with your reasoning, since the SMIME attachment is actually the real message (only encrypted) it probably makes more sense to read this message in your main library. I hope that inside that you will provide a convenient method to decipher the message and extract the attachments Thanks for your time Alex

bbottema commented 5 years ago

@gasparez15, can you tell me how you obtained the .eml file you posted in that zip? It contains more data than I get from the .msg parser (specifically ID, replyTo and bounceTo addresses as well as a bunch of Outlook original headers):

image

The most important one probably being the FROM name. Ignoring the minor fields, the following test shows the biggest problem:

Email emailParsedFromMsg = EmailConverter.outlookMsgToEmail("SMIME (signed and clear text).msg");
Email emailExpectedFromEml = EmailConverter.emlToEmailBuilder("SMIME (signed and clear text).eml")
        .clearId()
        .clearHeaders()
        .clearReplyTo()
        .clearBounceTo()
        .buildEmail();

EmailAssert.assertThat(emailParsedFromMsg).isEqualTo(emailExpectedFromEml);

image

gasparez15 commented 5 years ago

Hi Benny, that message was sent by Outlook (MS Office 2016) and received with Thunderbird. I saved it as an .eml file from Thunderbird

gasparez15 commented 5 years ago

This applies to the .eml file. About the .msg that has been saved by an application (outlook plugin) just before to be sent. To tell the truth, analyzing the situation now, I realized that maybe the .msg file is not fully complete, in fact opening it with Outlook, I didn't see the signature icon. Now I have downloaded it again from Outlook (sent messages) and the other copy of the file has the signature icon instead. I send you the two copies 2copies-of-sent-message-outlook.zip

bbottema commented 5 years ago

Hi Benny, that message was sent by Outlook (MS Office 2016) and received with Thunderbird. I saved it as an .eml file from Thunderbird

Aahhh, now I understand. Ok great, you helped me a lot with all this!

bbottema commented 5 years ago

But I still don't understand how this works.

First of all, you signed and encrypted a message, but not with a public key I gave you. So how is it that I can decrypt this message without any key? It's all supposed to work with keys and keystores and providers, but I haven't added any of that yet.

Second thing is, The Outlook message is parsed as a plain content message with subject, FROM and TO details and an S/MIME signed attachment. Is the resulting email supposed to be a combination/merge of the plain data and the decrypted message?

So:

Should result in the following?

It's still confusing me a lot.

gasparez15 commented 5 years ago

Hi Benny, the message is signed but not encrypted Message-security

bbottema commented 5 years ago

Hmm, the library I used reports it as encrypted, weird.

/edit: It's a bug in two libraries:

I fixed the bug in this library and raised a bug in the other.

bbottema commented 5 years ago

Hi @gasparez15, I finished implementation for reading signed .msg / .eml files in Simple Java Mail. Will be in the 6.0.0 release!

image

Note this doesn't support encrypted emails yet. Still working on that.

bbottema commented 5 years ago

I'm working on adding decryption now as well (only supports signed content currently). Are you able to provide me with additional examples?

I will need the following test set:

  1. signed (already have that one)
  2. encrypted
  3. signed, then encrypted
  4. encrypted, then signed
  5. maybe: some combination of multiple encryption/sign passes (should be rare in real world usage, but in theory this is possible and should be handled correctly as well)

For testing decryption, you would need to a generate private/public key pair, encrypt with the public key key and I would then need the private key to decrypt it.

bbottema commented 5 years ago

I wasted many hours fooling around with self signed pkcs12 certificates in Outlook and Thunderbird, but it seems I have no clue what I'm doing :/

Seems like I was going about it the wrong way. When I get some more time, I'll try the following guide: https://www.dalesandro.net/create-self-signed-smime-certificates/

Alternatively, I could try the shorter https://gist.github.com/richieforeman/3166387

/edit nope, didn't work in Outlook / Thunderbird...

bbottema commented 5 years ago

Got everything figured in the meantime (test data here), it was quite complicated due to the way the S/MIME spec tries to maintain backwards compatibility with older emails and Markenwerk's smime library having bugs...

But, I've finished implemented both encrypted and / or signed messages. Implemented in Simple Java Mail, will be in the 6.0.0 release.

bbottema commented 5 years ago

And now added signing / encrypting as well! Last step is to make sure it works from the command line as well and final documentation tweaks.