search

jakartaee / mail-api

Jakarta Mail Specification project
https://jakartaee.github.io/mail-api
Other
244 stars 101 forks source link

Cannot allow UTF-8 headers in MimeMultipart #371

Closed afelisatti closed 5 years ago

afelisatti commented 5 years ago

Consider the multipart content:

--the-boundary
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
Content-Disposition: inline; name="field1"; filename="£10.txt" 

yes
--the-boundary--

After parsing it through using MimeMultipart the pound symbol is altered (the boundary changes as we are echoing the data):

--d99d0400-1ff7-11e9-89ab-06f838e64be0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
Content-Disposition: inline; name="field1"; filename="£10.txt" 

yes
--d99d0400-1ff7-11e9-89ab-06f838e64be0--

This happens because the InternetHeaders used in MimeBodyPart to store the part headers are created with the allowutf8 option set to false so only ASCII chars are considered. I've validated while debugging that changing that to true results in no changes to the header. However, there does not seem to be a way of modifying that programmatically or even via a system property (I've tried mail.mime.allowutf8 but it didn't work and if it's meant to be in the session I can't find where to set that for MimeMultipart).

bshannon commented 5 years ago

First of all, that header is invalid. Headers are required to be ASCII. If you want to include non-ASCII characters in a parameter value, the parameter needs to be encoded per RFC 2231.

What mailer created this message? You should report this bug to the owner of that software.

Setting the Session property mail.mime.allowut8 might allow you to work around this buggy message, but be careful when setting it in a Session used to talk to a mail server that doesn't support the UTF-8 extension.

What version of JavaMail are you using? Are you reading that message from a mail server? Which one?

afelisatti commented 5 years ago

I agree it's a weird scenario but haven't found any strong RFC backed reason to simply disregard the issue entirely. This is actually an HTTP multipart message: we use MimeMultipart to parse the body. From my testing, clients like Postman can generate such content (just send any file with odd characters in their name). I wouldn't like to make UTF-8 handling the default but would like to at least allow users in the situation to configure it. The problem is I haven't found a way of doing so since MimeMultipart doesn't seem to take a Session nor does it expose a flag to modify the way the internal InternetHeaders for parts are handled. I've tested with version 1.6.3.

bshannon commented 5 years ago

RFC 2045-2047 are pretty clear that MIME messages are ASCII, that's why RFC 2231 exists.

HTTP is not exactly MIME, and relaxes some of the rules of MIME.

Postman isn't an email client, so it's not following the email rules.

You can try parsing an entire HTTP message as a MIME message using the MimeMessage constructor that takes a Session and an InputStream; that's where you can set the property. But this probably won't help since the property is only used to control the InternetHeaders object at the top level, so that address fields can contain UTF-8 characters. To change the handling of body part headers, you would need to subclass MimeMultipart and MimeBodyPart, and configure JAF to use your MimeMultipart subclass. That's probably more complicated than you want to consider.

Let me see if I can make some changes to JavaMail directly to support this.

afelisatti commented 5 years ago

Thanks, @bshannon! We can wait for a new release with support.

bshannon commented 5 years ago

I've pushed out a SNAPSHOT release with this change. Please try it and let me know if it works for you. You'll need to set the System property mail.mime.allowutf8 to true.

afelisatti commented 5 years ago

I just tried it and it works as expected. Is there a date for the 1.6.4 release? That way we can plan the upgrade accordingly. Thanks!

bshannon commented 5 years ago

Thanks for testing!

There is no date for the release, but I would guess it would be late Spring or Summer. It needs to synchronize with the Eclipse GlassFish 5.2 release, which is current unscheduled. If you need it by a specific date, let me know and I'll see what I can do.

afelisatti commented 5 years ago

I did find that our tests fail when run altogether which I assume is because the property is loaded only once, so the initial state remains throughout the run.

bshannon commented 5 years ago

Yes, it's loaded when the class is initialized.

afelisatti commented 5 years ago

Hi, @bshannon. Looking back on this, I was wondering if there's any timeframe for a release containing the fix. Thanks!

bshannon commented 5 years ago

My best guess is that it will be at least one month and more likely two months before the next release. We need to sort out some other things related to the overall Jakarta EE project before the release.

afelisatti commented 5 years ago

Hi, @bshannon. I noticed there are some preparations for version 1.6.4 going on so I wanted to check on this issue once more. Is there any ETA for when the artifacts will be ready on Maven central? Thanks!

bshannon commented 5 years ago

We're getting closer! We still need to go through the Eclipse reviews, which we hope will complete by early September.