Send outgoing text/plain as format=flowed

[Valodim]

Outgoing text/plain mail can be formatted as format=flowed according to rfc2646.

We already have a method in FlowedMessageUtils from an the apache-licensed Apache James project, so this should only be a matter of calling the right method at the right place.

Starting point should be around MessageBuilder.buildBody

[AnirudhBadri]

I am a beginner to the open-source community and would like to start with this bug. Can I go ahead?

[danwain]

I have submitted a pull request with my take on the feature. Any feedback would be welcome. #2305

[cketti]

We currently always use quoted-printable as transfer encoding. However it shouldn't be used with format=flowed.

Right now I don't think we gain anything by using format=flowed. It's unnecessary when using quoted-printable. And the spec notes that it might cause some problems with right-to-left languages.

[Valodim]

Ah, you're right. Sorry, I didn't see that @danwain

[ghost]

In Thunderbird format=flowed is default and is used. It only uses quoted-printable for pgp messages. A lot of the email I send isn't actually encrypted.

I have found wrapping works reliably with format=flowed particularly on mobile devices.

[benbucksch]

@cketti , @Valodim : Format=flowed solves a number of real world issues with plaintext. It not only allows wrapping lines properly and adapting to different screen sizes (which is particularly important for mobile clients like K9Mail) and avoids "comb wrapping", but also specifies clearly how quotes should look like, unambiguously fixes the mbox "From at the start of the line" issue in a reversible way etc..

format=flowed is essentially plaintext mail done right.

Please note that you already do implement format=flowed (RFC 2646), in FlowedMessageUtils.java.flow().

Ben (Thunderbird developer who implemented a lot of the plaintext processing, but not the format=flowed part, and thinks that format=flowed solved a lot of issues and saved me hairs.)

[ghost]

Shouldn't this issue be opened again?

[shader]

Coming from #7084, I would also like to reopen this issue.

https://useplaintext.email/ includes "Wraps text or format=flowed" as a criteria for clients supporting plaintext, and K-9 currently does not meet that requirement.

With some basic testing, it seems that indeed K-9 is not wrapping the text being sent, which is improper formatting for text emails. The 'soft breaks' in quoted-printable are an encoding-level feature, not real line breaks, resulting in long lines that some clients do not wrap.

To reiterate the points from @benbucksch and others:

• for users who care about plain text emails (rare, I know), proper wrapping may be important
• plaintext emails should be wrapped at ~72 lines
  • K-9 could get a recommendation at useplaintext.email if handled correctly
• format=flowed improves line wrapping flexibility while maintaining backward compatibility
• the code is already in the code base, and K-9 already supports reading format=flowed emails, so why not support sending it?

I am willing to do the work to support this, I just want to reopen discussion so we can figure out the best way to fit it into your existing design goals. At the very least I think it could be an option in the "Sending mail" settings; I think the question is how you think that option should be presented.

Some ideas:

• an alternative plain text format (in the same drop down as Plain Text, HTML, and Auto)
• as a separate dropdown for choosing which plain text format to use

[cketti]

No modern email client treats long lines in text/plain parts to mean "this is a very long line, don't wrap it for display purposes; scroll horizontally if you have to". Since the introduction of Content-Transfer-Encoding (that can be used to create short encoded lines, satisfying the line length limits and recommendations in RFC 5322) using format=flowed is obsolete.

Soft line breaks in format=flowed are no different from soft line breaks in quoted-printable. They are part of the encoding, not the content. The only thing format=flowed makes explicit is "long lines should be wrapped when displaying the text". But as already mentioned, this is also done without using format=flowed.

Case in point: The email notification for the comment above contains the following text/plain part.

Content-Type: text/plain;
 charset=UTF-8
Content-Transfer-Encoding: 7bit

Coming from  #7084, I would also like to reopen this issue.

https://useplaintext.email/ includes "Wraps text or format=flowed" as a criteria for clients supporting plaintext, and K-9 currently does not meet that requirement.

With some basic testing, it seems that indeed K-9 is not wrapping the text being sent, which is improper formatting for text emails. The 'soft breaks' in quoted-printable are an encoding-level feature, not real line breaks, resulting in long lines that some clients do not wrap.

To reiterate the points from @benbucksch and others:
- for users who care about plain text emails (rare, I know), proper wrapping may be important
- plaintext emails *should* be wrapped at ~72 lines
  - K-9 could get a recommendation at useplaintext.email if handled correctly
- `format=flowed` improves line wrapping flexibility while maintaining backward compatibility
- the code is already in the code base, and K-9 already supports reading `format=flowed` emails, so why not support sending it?

I am willing to do the work to support this, I just want to reopen discussion so we can figure out the best way to fit it into your existing design goals. At the very least I think it could be an option in the "Sending mail" settings; I think the question is how you think that option should be presented.

Some ideas:
- an alternative plain text format (in the same drop down as Plain Text, HTML, and Auto)
- as a separate dropdown for choosing which plain text format to use

It contains long lines. All clients that I tested wrap the lines when displaying the message. All without using format=flowed.

Support in email clients for quoted-printable enabling arbitrarily long (decoded) lines is virtually ubiquitous. Support for format=flowed is not. It also has weird edge cases and is non-trivial to implement (see e.g. https://github.com/apache/james-project/pull/975). There's no good reason to use it when sending messages.

The only advantage of format=flowed is that it creates output that can be used with email clients that don't support proper wrapping when displaying a message. Those should be considered broken since the 1980s.

[benbucksch]

format=flowed is obsolete

Format=flowed also specifically defines how to unambiguously mark quotes, and how to preserve flowed text in quotes.

1. Where does quoted printable define what a quote is? Or which other IETF standard defines it? Where in the standards is defined how the combination of quotes and flowed text works?
2. You said that all "modern" email clients implement quoted-printable and flowed text correctly. Do they also all, without exception, a) display and b) send quotes correctly? Including quotes of quotes and flowed text in quotes? 100% all of email apps? Or are there email apps in significant use which have shortfalls there? Which then affect everybody else, including K9mail users?

The advantage of format=flowed is that it works with all clients. Email has many vastly different implementations, and only 1 sub-par email app replying destroys the thread for everybody.

[belkka]

@cketti

Soft line breaks in format=flowed are no different from soft line breaks in quoted-printable. They are part of the encoding, not the content.

Actually, format=followed is a media type rather than encoding (meaning it's different than quoted-printable). Compare:

Content-Type: text/plain; format=flowed

Content-Transfer-Encoding: quoted-printable

I emphasize this because format=flowed allows to mix different chunks of text — flowed (human text) and fixed (code chunks/git patches). To be precise, it defines fixed and flowed lines and then defines a paragraph (4.1):

A series of one or more flowed lines followed by one fixed line is considered a paragraph, and MAY be flowed (wrapped and unwrapped) as appropriate on display and in the construction of new messages

Also (4.2):

Any number of fixed lines can appear between paragraphs.

I can think of it as of "light-weight" markup (slightly rich text), where only semantic equivalent of HTML <pre> or Markdown code blocks are allowed.

As I understand, advanced "plain-text-email" users (lets call them hackers) actively work with code blocks and git patches in email message bodies (https://git-send-email.io/). Since format=flowed is not widely adopted, they have to deal with usual (not flowed) text/plain a lot.

No modern email client treats long lines in text/plain parts to mean "this is a very long line, don't wrap it for display purposes; scroll horizontally if you have to"

True, but my guess is, plain-text-hackers intentionally disable soft-wraps in usual (not flowed) text/plain emails (even if their modern clients are capable of doing soft wrap), because it's more important for them to review included patches (code blocks). Therefore even if I use K9 mail to send a long personal pure-human-text email to a hacker, they still have soft-wraps disabled in their global settings for usual (not-flowed) text/plain, which makes my email unreadable for them. With format=flowed I have a power to explicitly indicate that soft-wraps should be enabled for my message. Even more — I can mix different types of wrapping, not just toggle it for the whole message. Even if recipient doesn't recognize format=flowed, they will see a readable version of my email (no long paragraph-lines).

Possible solutions

1. Lightweight solution, no UI changes

Since K9 Mail is an Android app, it's more reasonable to assume that users do not send any pre-formatted plain text chunks (code blocks/patches) at all, rather than whole email is pre-formatted (fixed) text/plain. In other words, it may be reasonable for the client to send text/plain; format=flowed and silently assume that every input line is a flowed paragraph (in terms of RFC3676). It's impossible to send fixed lines between paragraphs (code blocks)

2. "Perfect", complex solution

Implement some additional UI to allow creating "semantic equivalents" of HTML <pre> blocks in composed text/plain messages. They are sent as sequences of "fixed" lines between "paragraphs" of format=flowed as described in rfc3676. The solution requires maintaining more complex internal representation when composing emails ("slightly rich text").

3. Middle ground, little UI change, but less user-friendly

Add a "toggle format=flowed" option either in global settings or email editor. The main consequence of enabling this option is switching Content-Type: text/plain header to Content-Type: text/plain; format=flowed. The experienced user (aware of rfc3676) is responsible for typing spaces where they want to define a "flowed" line. K9 mail then (optionally) "decodes" and "re-encodes" format=flowed in order to re-balance lengths of flowed lines (for unaware recipients, mailing lists etc). This approach may be easier to implement if we can find and reuse existing libraries for "decoding" format=flowed -> "slightly rich text". Otherwise there is no advantage over solution (2)