gmailieer: behave like sendmail

[flokli]

sendmail(1) allows additional recipients to be passed in the command line. The path to the message is never passed, it's always stdin:

sendmail [option ...] [recipient ...]

At least neomutt makes use of the recipient arguments, so let's have lieer behave properly.

Contrary to SMTP, it seems recipients can only specified inside the message body when interacting with GMail. If additional recipients are passed, we add them to the Bcc field.

Fixes #152

[gauteh]

eml is not actually used when sending, it is just used for reading parameters to group the message with the correct thread, so setting any parameters is not going to have any effect at the moment. I was hoping to avoid processing the input message at all, is there any way to tell neomutt to set the headers? We could validate that any addresses supplied are also present in the headers (but then we do not need to modify the raw message).

[flokli]

neomutt makes use of sendmail(1)'s interface here - and usually the recipients in the To:, Cc:, Bcc: fields can be something entirely different than the address you want to send a message to - the recipients passed to sendmail are effectively what RCPT TO is in SMTP, which isn't part of the message header or body.

This is interesting for Bcc, or "bouncing over emails" (not sure if possible with notmuch-emacs)

I'm not sure if there's a way to separately specify recipients in the Gmail API, but from a quick look, it doesn't seem to be the case?

Simply ignoring recipients in lieer send might silently break sending mails with Bcc (if write_bcc is set to the default), and break bounces. Adding this into the Bcc: field might be a workaround (If we pass over the modified eml, of course). I didn't yet check how Gmail would handle a "bouncing emails" event in that case, though.

Another option would be to use gmails smtp frontend via msmtp and OAUTHBEARER authentication- but we'd need to extend the scopes for that.

What do you think?

[gauteh]

Florian Klink writes on May 14, 2020 10:51:

neomutt makes use of sendmail(1)'s interface here - and usually the recipients in the To:, Cc:, Bcc: fields can be something entirely different than the address you want to send a message to - the recipients passed to sendmail are effectively what RCPT TO is in SMTP, which isn't part of the message header or body.

How often does it happen that the fields do not match? I am wondering if it would work to not support that case, but validate whether they do match (if supplied on cmd-line).

This is interesting for Bcc, or "bouncing over emails" (not sure if possible with notmuch-emacs)

I'm not sure if there's a way to separately specify recipients in the Gmail API, but from a quick look, it doesn't seem to be the case?

No, it does not seem to be possible.

Simply ignoring recipients in lieer send might silently break sending mails with Bcc (if write_bcc is set), and break bounces. Adding this into the Bcc: field might be a workaround (If we pass over the modified eml, of course). I didn't yet check how Gmail would handle a "bouncing emails" event in that case, though.

So the only recipients the email should actually be sent to are the ones specified on the command-line? In that case adding them would cause both the command-line recipients and the recipients specified in the header to receive the email (when using the GMail API). This is probably not the intended behavior!

Another option would be to use gmails smtp frontend via msmtp and OAUTHBEARER authentication- but we'd need to extend the scopes for that.

What do you think?

I would prefer to keep it simple and use the Gmail API. The only reason for doing this through lieer and not using msmtp directly is to avoid having to set up a password for the application?

[flokli]

neomutt makes use of sendmail(1)'s interface here - and usually the recipients in the To:, Cc:, Bcc: fields can be something entirely different than the address you want to send a message to - the recipients passed to sendmail are effectively what RCPT TO is in SMTP, which isn't part of the message header or body.

How often does it happen that the fields do not match? I am wondering if it would work to not support that case, but validate whether they do match (if supplied on cmd-line).

We could do that. We should provide a somewhat useful error message linking to a wiki page explaining the common pitfalls (see below)

This is interesting for Bcc, or "bouncing over emails" (not sure if possible with notmuch-emacs)

Bouncing mails won't be possible

I'm not sure if there's a way to separately specify recipients in the Gmail API, but from a quick look, it doesn't seem to be the case?

No, it does not seem to be possible.

Bummer :-/

Simply ignoring recipients in lieer send might silently break sending mails with Bcc (if write_bcc is set), and break bounces. Adding this into the Bcc: field might be a workaround (If we pass over the modified eml, of course). I didn't yet check how Gmail would handle a "bouncing emails" event in that case, though.

So the only recipients the email should actually be sent to are the ones specified on the command-line? In that case adding them would cause both the command-line recipients and the recipients specified in the header to receive the email (when using the GMail API). This is probably not the intended behavior!

I agree. I was hoping GMail wouldn't duplicate if it already saw that Message-ID, and only send to new Bcc, but that was more of a wild guess

• we shouldn't modify message bodies after the fact, and being able to specify additional recipients in "bouncing" cases should be supported IMHO - it's already supported by SMTP, too

Another option would be to use gmails smtp frontend via msmtp and OAUTHBEARER authentication- but we'd need to extend the scopes for that.

What do you think?

I would prefer to keep it simple and use the Gmail API. The only reason for doing this through lieer and not using msmtp directly is to avoid having to set up a password for the application?

I can't use Application passwords, as they're disabled in that specific GSuite. To use Gmail with Oauth, I'd need to create an entirely new app with these scopes, get approval, to then use the access keys via something like getmail-gmail-xoauth-tokens, which is quite some setup. Being able to just do it with the token already obtained for lieer anyways would probably be easier - of course, it'd need to be a part of lieer, so requesting these scopes is justified.

[gauteh]

Florian Klink writes on May 14, 2020 12:11:

neomutt makes use of sendmail(1)'s interface here - and usually the recipients in the To:, Cc:, Bcc: fields can be something entirely different than the address you want to send a message to - the recipients passed to sendmail are effectively what RCPT TO is in SMTP, which isn't part of the message header or body.

How often does it happen that the fields do not match? I am wondering if it would work to not support that case, but validate whether they do match (if supplied on cmd-line).

We could do that. We should provide a somewhat useful error message linking to a wiki page explaining the common pitfalls (see below)

If this is an acceptable solution, I think we should do that. This is as far as we can get within the GMail API.

A future extension to using SMTP in stead is something that I would like to avoid, if possible, but I am not totally against it.

[gauteh]

@rbutoi: You are perhaps interested in this discussion.

[flokli]

I'll push something later today checking recipients, but I'd really be interested in the continuation of the discussion.

Bouncing over a message thread to someone is something I use quite often (for example, bouncing over a mailing list thread to another person that didn't subscribe to the archive before), and it'd be a bummer to not do this with GSuite.

[gauteh]

Florian Klink writes on May 14, 2020 13:16:

I'll push something later today checking recipients, but I'd really be interested in the continuation of the discussion.

Bouncing over a message thread to someone is something I use quite often (for example, bouncing over a mailing list thread to another person that didn't subscribe to the archive before), and it'd be a bummer to not do this with GSuite.

Yeah, it seems the best way to achieve that now is to clear recipients and Bcc:. Or you have to forward (to get the context). But it is not the same, I agree. Bouncing is not really something that seems to be common in more 'modern' mail clients. Either of the above workarounds is probably something that should be done by the MUA and not by lieer.

[flokli]

I did think about this a bit and pushed a version that should provide some nice error messages to https://github.com/gauteh/lieer/pull/154, PTAL.

[flokli]

I'll close this PR, #154 is the way to go IMHO.