Subject and Group names

r10s commented 6 years ago

I am wondering if we can improve the usage of the e-mail "subject" and make it more conform with standard MUAs.

Currently, incoming Subject: headers are used for the following:

the subject is shown before the message if it is not prefixed by Re: etc. and if the message comes from a normal MUA
for ad-hoc-groups, the subject is used as the initial group name (Re should be stripped, but currently isn't)

For outgoing messages, the Subject: header is used as follows:

It is set to an excerpt of the message prefixed by the word Chat: and possibly the group name.
The incoming subject is not used

Ideas that came into my mind but need more thinking and discussion:

set the expected subject for outgoing messages, so for a group name "Family", the subject will be "Re: Family", this would make sorting things easier for existing MUAs, otoh the text excerpt will be missing. At the latest when we decide to support Mailing lists, sth. like that should be done, so maybe it is better to do the same for all types of mails.
For the view from the Delta Chat side and for Delta-Chat-to-Delta-Chat this won't change anything
group names are currently handled by the new Chat-Group-Name: header, as a result normal MUAs cannot change the group name. Maybe we should just treat the subject (stripped by Re: etc.) as the group name? This would allow other MUAs to change the subject as well. However, users of normal MUAs just changing the subject may not be aware of the consequences, so maybe this is no good idea.
should updated subjects change the group name? This may be useful but maybe also unexpected if a group just changes its name and the user cannot find it again. Moreover, it is no easy job to strip out the Re: Fwd: stuff when regarding all languages.
But maybe it should be possible by groups initially created as Ad-hoc groups? Maybe at least if the name was not changed by Chat-Group-Name?
The Subject is currently not used to create groups - this is only done by the existing member list. So, two mails with two different subjects to the same recipients will be filed under the same Ad-hoc group. We could create different groups for this, however, I fear this would spam the chatlist and tear apart chats. I also do not feel comfortable to give such a power to the subject. Maybe the existing approach is just fine.

I am sure I've forgotten something an I am looking forward to comments and suggestions.

testbird commented 6 years ago

Don't mailinglist subjects often look like Re: [deltachat-dev] Subject and Group names? Then maybe the string between square brackets could maybe serve as the group name (and even be editable for peer to peer recipient lists).

testbird commented 6 years ago

it is no easy job to strip out the Re: Fwd: stuff

Square brackets may work better.

two mails with two different subjects to the same recipients will be filed under the same Ad-hoc group

For now, it should be fine not to support special topic chats. If only considering the square brackets, the risk is lower anyway, and the only result would be two subsequent renames. Something the group should be able to discuss and can be easily corrected by any group member.

testbird commented 6 years ago

For now, it should be fine not to support special topic chats.

I am having second thoughts about this for mailing lists. Messages may not be readable/nice at all out of context if DC is mixing up different threads from mailing lists into a single group chat. So maybe it is better to create special topic [list-nick] thread topic chats at least for managed mailing lists as they tend to have more messages and classical MUAs communicating.

For peer2peer recipient list groups, the condition to start a new special topic chat might be a subject that has a string behind the [group nick] that is not equal to the beginning of the body.

Edit: ... And not equal to the previous subject? (as with classic MUAs)

Edit 2: Heck, if supporting special topic chats, it should also be possible for one to one chats. So even if no [group nick] is in the subject.

Nota bene: How do you think about these features? I really like the discussed features, but think they would open the next can of ~~worms~~bugs, just too early. For all of these things it is also important to have testers etc., and some of the current issues already completely prevent part of the users from continuing to use and test deltachat. (Think of chat noise and flood, due to folder creation errors.) So IMHO fixing some current issues (once and for all) before working on further features, I think, would be appropriate and thankfully appreciated.

r10s commented 6 years ago

but think they would open the next can of ~~worms~~bugs, just too early.

I agree. We should really keep things simple for now; the spec issues are there to remove complexity, not to add some ;)

r10s commented 6 years ago

for ad-hoc-groups, the subject is used as the initial group name (Re should be stripped, but currently isn't) [...]

should updated subjects change the group name? This may be useful but maybe also unexpected if a group just changes its name and the user cannot find it again.

Just brainstorming: Another idea to get around this problem: Do not use the Subject for the ad-hoc group name but the names of the members, So if the ad-hoc group members are Alice Adams <alice.adams@foo.bar>, Bob Bärbel <bob.baerbel@foo.bar> and <chris.cars@foo.bar>, the default group name may just be Alice, Bob, Chris

Advantages:

as we create the ad-hoc groups by the group list and not by the subjects, it reflects more what we're doing
the subject can be changed without affecting the group name
the group name can be changed as for any other group and is independent from the subject
it is more obvious that the chat is a group chat

Disadvantages:

normal MUA cannot change the group name (however, they also cannot do this for non-ad-hoc-groups)
the name does no longer match when adding/removing members
The naming approach with a comma-separated list does not work well with large usergroups or if there are no names associated with the e-mail address

(for missing names we could use the part before the @ then, which might be an good idea in general. for larger groups we could use sth. like Alice, Bob +3 (localisation would disturb unique group names here, also we should take only to use the names directly from the mail starting the ad-hoc-group (not the edited ones or seen on other mails)))

testbird commented 6 years ago

We should really keep things simple for now; the spec issues are there to remove complexity, not to add some ;)

I think the brainstorming is good also to keep things simple in the future (by implementing good minimal subset, and not something too simple that needs to be reworked later).

Another idea to get around this problem: Do not use the Subject for the ad-hoc group name but the names of the members

You made me realize mixing both subjects and recipients in the chats list is inconsistent and could likely crate trouble later. To cleanly separate subject threads, the the subjects could be something like tabs or accordions within the chats. I believe supporting email threads could be a very good feature some day. So not mixing subjects into the group name, seems to be right to me.

I don't see a problem with non-DC clients not being able to change the exchanged group name. They would name the distribution lists independently, if at all, there would have to be some vcard? standard for how MUAs name and exchange distribution lists.

Since the default name is hardly the ultimately desired one, the members can change, and the name is changeable, it might also be pragmatic to just start with a default like a date/timestamp or simply "new group (2)".

hpk42 commented 6 years ago

On Fri, Mar 09, 2018 at 23:47 -0800, testbird wrote:

We should really keep things simple for now; the spec issues are there to remove complexity, not to add some ;)

I think the brainstorming is good also to keep things simple in the future (by implementing good minimal subset, and not something too simple that needs to be reworked later).

seconded.

Another idea to get around this problem: Do not use the Subject for the ad-hoc group name but the names of the members

You made me realize mixing both subjects and recipients in the chats list is inconsistent and could likely crate trouble later. To cleanly separate subject threads, the the subjects could be something like tabs or accordions within the chats. I believe supporting email threads could be a very good feature some day. So not mixing subjects into the group name, seems to be right to me.

"tabs" or "accordions" within a chats sounds like an interesting but also major UI change.

If i see it correctly, then the issue of ad-hoc groups arises when a non-DC client sends mail to multiple recipients, and one (or more) recipients use DC for reading it.

To begin with, the DC recipient must then have added the sender once under "contact requests", to see the message in a DC chat, right? If so it's not clear that adding a sender under "contact request" implies that DC "owns" all messages from that sender -- particularly group-messages like we are discussing here might warrant a "second" contact request. or maybe one needs to enable an option "create ad-hoc groups for group-mails from non-DC clients" first? I generally suggest to rather drag less messages to DC by default than more. It's better if users come positively complaining "i want to see these messages with Delta" rather than getting annoyed at seeing things with DC they don't expect or don't want to see. At least i've have heart about the latter annoyance several times.

Then, if the sender sends another mail to similar or same recipients but with a different subject i guess it should show as another chat-group. This does not need to be based on parsing the subject (which is fragile anyway) but better be based on the "reference" headers ("in-reply-to|references"). If the DC reader has two chats for two different threads it means it can answer and the non-DC clients will see it thread appropriately. So i think in this "show group messages from non-dc clients" mode it makes sense to distinguish by looking at the references.

If we just base groups on participant addresses, this would probably not allow separating two separate threads. So i think just taking the participant identities ("alice, bob, ...") is not distinctive enough and also it's misleading if more people get CCed along the conversation. Morever, the group already has the member list and so duplicating it in the group name is redundant (and possibly confusing). So I think it makes sense to use the subject from the top-most mail we have seen.

testbird commented 6 years ago

"tabs" or "accordions" within a chats sounds like an interesting but also major UI change.

It came to my mind only from translating a MUA like behaviour, but for a first implementation, and even UI wise "special subject chats" within the chat list seem more straight forward.

Then, if the sender sends another mail to similar or same recipients but with a different subject i guess it should show as another chat-group.

Yes, I think so, too. It's pragmatic and easy to understand.

This does not need to be based on parsing the subject (which is fragile anyway) but better be based on the "reference" headers ("in-reply-to|references").

Maybe it could be a good compromise, to only open another chat if no/or a new reference is made? (Just as classic MUAs usually keep changed subjects within a thread, and only start a new thread if there is no in-reply-to match?)

As DC clients usually don't conserve a running subject, it would also need some way to explicitly start a new (special subject) chat with the same recipients as an existing group chat.

If the DC reader has two chats for two different threads it means it can answer and the non-DC clients will see it thread appropriately.

Yea. To let non-DC clients show threads appropriately, maybe let DC send answers in group chats always with proper in-reply-to references "in thread".

If i see it correctly, then the issue of ad-hoc groups arises when a non-DC client sends mail to multiple recipients, and one (or more) recipients use DC for reading it.

This may actually also provide the room to solve the unwanted ad-hoc group creation in cases like https://github.com/deltachat/deltachat-android/issues/267#issuecomment-372031372 (DC responding to DC message that was forwarded with an additional CC: or To:). IIUC, the too permissive ad-hoc group creation could then be prevented by disabling ad-hoc group creation for messages from Email-Chat compliant (DC et. al.) clients?

testbird commented 6 years ago

To let non-DC clients show threads appropriately, maybe let DC send answers in group chats always with proper in-reply-to references "in thread".

And for long-ongoing one-to-one threads DC could possibly only optionally break ongoing chats into a new thread by answering with a new "first" message without a in-reply-to reference? (every x messages and y days?)

Edit: This aspect is probably getting solved with https://github.com/deltachat/deltachat-core/issues/131

WinAuthFan commented 6 years ago

The way the Subject string is implemented is so confusing. Subject should always be just Chat: [first few words of the message].

The grouping of messages should not rely on the Subject. In deltachat, this is already implemented using headers anyway. Don't the other MUA's also use the In-Reply-To header to group messages?

Putting the group name anywhere in the email, including in the Subject, is bad. First, deltachat doesn't even tell you that group names are put into the subject. As a minimum, it should tell you that. What if you use a funny name for a bunch of your friends -- then they all find out when you send them a message.

Second, the words Chat: [group name] take up most of the visible Subject in many MUA's. So the Subject, especially the [first few words of the message], becomes irrelevant.

Let's think through what we're trying to achieve.

In deltachat,
- Subject text is irrelevant
- Message grouping is implemented using headers
In other MUA's
- Just using Chat: [first few words of the message] as the Subject is consistent with the ethos of chatting -- the reader can see right away what you are messaging them about.
- Message grouping is implemented using headers, isn't it?
- Even if a MUA does not group messages using headers, so what's the big deal?

testbird commented 6 years ago

Even if a MUA does not group messages using headers, what's the big deal?

I guess it's to make group messages quickly recognizeable. To avoid surprises if clicking "answer"?

WinAuthFan commented 6 years ago

@testbird DC should try to play nice with other MUA's. However, if other MUA's are not working correctly, DC cannot be responsible for that. All MUA's should use the headers to figure out threads. If they don't, then it's their problem that they have to fix. K-9 groups emails into threads correctly using headers.

testbird commented 6 years ago

I think the group or recipient list names in the subject are mainly for the user (me) to associate the threads, not for the MUAs grouping, e.g. -[Deltachat] running again today versus -[Sporties] running again today.

testbird commented 6 years ago

Just in case it wasn't clear, the "-chat"-brackets -[] in my proposed solution are to be part of the subject (not demarking substitutions) There is also https://github.com/deltachat/deltachat-core/issues/49

WinAuthFan commented 6 years ago

We are really overthinking this. We should not try to find the "best" solution. Instead, we could let the user decide. #239

testbird commented 6 years ago

I believe the modern "mobile apps" with limited UI actually do require to think up to the best solutions, plus, allow the user to specify (delimit) a custom subject (e.g. with - as in received emails).

I think the idea that I outlined in response to https://github.com/deltachat/deltachat-core/issues/239#issuecomment-413578420 could lead to a good general solution, that could just work without much UI and quite intuitively even. (delimiter, subject preserving, option to save a default draft globally and per chat)

tapete commented 5 years ago

My 50 Cent:

Do not use the prefix Chat or Group
Insert "change subject" button in groups and chats. With this button, the user can change the subject of the outgoing emails. If the user does not change the subject, the current subject style is used but without the prefix.

This would make the app more compatible to other mail clients. The current subject style is the main reason I am not using delta chat all the time as I dont want to confuse my chat partners.

r10s commented 5 years ago

@tapete +1 from me, apart for the "change subject" button. i do not want the users to think about this - esp. as it is only important if the recipient is using non-delta MUAs - a think the sender might not even know and may start thinking about the subject with every single post. however, the subject can be set indirectly if we just use the group name here.

tapete commented 5 years ago

@r10s

i do not want the users to think about this

I agree, so I suggest that the button is on the upper site of the chat window or hidden in the menue on the upper right. The user can set a subject once and all messages will have this subject. Or the user ignores the button and a standard subject is created. So the user does not need to decide for a subject each message he is writing.

esp. as it is only important if the recipient is using non-delta MUAs

100% of my contacts use non-delta MUAs and I think this will be the case for a lot of users. That is why I think it is important that delta chat should work perfectly with thunderbird and co.

however, the subject can be set indirectly if we just use the group name here.

Yes. At the moment the subject of Groups look like Chat: Great Group: Hello friends... "Hello friends..." is dynamic and is different for each messagen. This makes it hard for non delta chat users when they try to sort messages by subject. It would be more compatible if the subject of groups look like: Great Group

testbird commented 5 years ago

@r10s -[] Having to change the group name to define the entire subject seems to be the wrong context and too inflexible to me. -[] And it may also not work so well for one-to-one chats.

@tapete About specifying the subject like this - It could be possible to specify (delimit) a custom subject just by using the "minus" ( -) when writing, to delimit the subject. Exactly as the subjects of received email messages are shown in deltachat. (This would not require any UI, or the user to always think about the subject, but would still allow further UI features where wanted.)

Another idea is to shorten the default subject, to not contain the english-only word "Chat:" but use brackets that may also be seen to kind of symbolize "speech bubbles", as in the examples above.

For groups, the brackets may: -[contain the group name if applicable] Followed by the beginning of the message text ... see https://github.com/deltachat/deltachat-core/issues/239#issuecomment-413578420. And ending the subject with ... or | to indicate whether there is, or is no further text in the message body.

tapete commented 5 years ago

@testbird

What would not to be to your liking if it would be possible to specify (delimit) a custom subject just by using the "minus" ( -) when writing, exactly as the subjects of received email messages that are shown in deltachat?

This would be a simple solution but it is not self-explanatory so the user must have heard of this feature to be able to use it. So in my opinion a menu entry might be a feasible way. I mean the menu at the upper right which is accessible by the three dots. What do you all think about it?

testbird commented 5 years ago

Of course discoverability would be good, but I think the devs consider subject customization an advanced feature and don't want to add UI elements.

Nevertheless, the quick and convenient, in-line, text based subject writing idea could be extended to be visually discoverable:

With a UI button that inserts a pre-selected text into the editor field like: "Enter subject| - ${previous text if there was one}"
And / or by displaying a (grey) - where the automatic subject will end, if there is no manual separator.

testbird commented 5 years ago

And another, possibly even more self explaining UI idea (not only for a separate "contact requests" e-mail inbox tab) may be:

~~The edit area may automatically change into a two-line or two-field view (subject and body) as soon as the entered text exceeds the max. subject length, to visualize the resulting email formatting.~~

The DC user interfacce may, for example, just use a soft shade (background) for the text that goes into the subject (ending with a shaded separator -).

And a simple first manual Enter in the text could always allow to manually switch into the "body" field (an earlier new-line (carriage return) allows to use a shorter manual subject).

testbird commented 5 years ago

-[Forum Discussion] finding the right solution: https://support.delta.chat/t/subject-of-emails/264 |

testbird commented 5 years ago

User pcrockett posted the good idea to show customized subjects in the chat's title bar, which also allows for full integration (e.g. for proper mailing list support), by listing all used subjects in a chat's context tab (i.e. like the available photos and documents etc.) and to filter the messages listed in the chat view according to a selected subject. https://support.delta.chat/t/subject-of-emails/264/74

deltachat / deltachat-core

Subject and Group names #128