Forwarding

[mattfbacon]

I noticed that this is on the roadmap but I'd like to work on implementing it. I've looked into the code and I have a decent idea of how to get started, but I have a couple questions:

• Should forwarded email be subject to the same spam checks and whatnot?
• Should the forwarded emails be retained somewhere in the store?
• How much of this sending logic should be shared between submit and this new forwarding function? Or can forwarding actually use submit directly?

[mjl-]

hi matt, good to hear! i've postponed forwarding because it turns out a bit more involved than it appears on first glance. so we'll have to think it through thoroughly. indeed spam filtering, and what to do with non-deliverable emails, especially need attention.

you may have already found related forwarding-discussion in https://github.com/mjl-/mox/issues/57. i'm away from keyboard for another day, but will write up some thoughts here soon.

[mjl-]

Took me a bit longer to get back online, sorry. Below is a brain dump of my thought process, ending in a potential conclusion.

Incoming email (delivery) is normally delivered to a local account. Several checks, including the junk filter may reject an incoming message, based on:

• dmarc policy and alignment: a "message From" user@domain.example will be rejected if the domain has a dmarc policy of "reject" and the domain in the "message From" address doesn't match the spf-verified domain in the "smtp transaction MAIL FROM" address and doesn't have a matching dkim signature for the domain in the "message From" address.
• Reputation of sender address/domain/dkim/spf/ip(exact/narrow/wide netclass). All based on previous classifications as junk or nonjunk in the account.
• Analysis of the content, based on previous (non)junk classification of messages (used as training).
• Optional dns block list ("dnsbl", currently configured for entire mail server, not per account, though I've been thinking of making it per account).

Reputation- and content-based analysis is done only based on classifications in the account, not with classifications from other accounts.

About forwarding: I'm assuming you want to create an address and forward incoming email to another address. Perhaps just as an alias. Or perhaps in the future when a user leaves domain and mail server and would like to have emails forwarded to their new address.

Sketching a forwarding scenario: Let's say sender with address A is sending a message to address B, which is configured to forward to address C. And A, B and C all have different domains (if B and C are on the same domain, things would be easier). A, B, C could all be hosted on different mail servers, running diffent software, each with different behaviour. Below I'm using "domain A" to mean the domain of address A, and "server A" to mean the server that sends/accepts email for address A, etc.

What would B do for an incoming message from A?

• With dmarc/reputation/content/dnsbl-based checks, possibly reject the message. For analysis to work fully, we would need messages with (non)junk classification in the account. Rejecting a message can be done with a permanent 5xx response, or a temporary 4xx response. With a 4xx response, server A will retry for a while, but should get the same reject until it eventually gives up.
• Otherwise, accept and add message to the queue for delivery to C. The message content (headers and body) will be queued mostly unchanged, only some hop headers would be added, like Received, arc and possibly dkim. When sending, we have a choice for which address to use in the "smtp transaction MAIL FROM":
  • If we keep address A, we will likely cause C to evaluate spf to "fail" because the IP address of server B isn't in domain A's spf records. This may cause dmarc to evaluate to "fail" at server C, causing it to reject the message.
  • Use address B. This would cause C to evaluate spf to "pass", but may cause dmarc to evaluate to "fail" at server C because the "message From" address has a different domain than B (addresses are not aligned). This would happen when there is no valid aligned dkim signature to cause a dmarc "pass".
  • Use an address made up with a scheme that has the domain of B, but with the original address A encoded in the localpart. Like the previous option, but may cause a late/non-immediate reject/bounce to be detected as a failure to forward (but see below, where we try to avoid this).

It doesn't sound great to store all messages for C on server B (takes up storage, may contain sensitive information). It also would only be helpful if we get (non)junk classifications done at C back to B, but there is no clear path to do that. B could perhaps interpret a reject by C as a junk classification. That may cause the second and later message from a junk sender to be blocked. But for first-time senders with junk (most of them?), it wouldn't help.

What would C do for an incoming message via B originally from A?

• If C is running mox, the user may have configured that messages from B are recognized as forwarded messages, and accept all such forwarded messages based on the assumption that B has done junk filtering. But as we've seen above, B may not have enough information to decide on junkiness. So that would mean C will get all the junk and can never push back on unwanted senders (which is useful for supposedly legitimate senders that won't take you of their list).
• If C is running other software, or user C has not configured messages from B as being forwarded messages, junk filtering at C would apply: dmarc and sender address/domain/dkim/spf/ip checks. The result:
  • dmarc: Since dmarc could no longer "pass", a non-junk message may now be rejected. This is unacceptable, and means you can only forward to mail servers that don't enforce dmarc policies. Either at all, or for messages identified as forwarded through B; For me it's an open question how this is normally configured in other mail servers.
  • Likewise, server C shouldn't do ip-based junk analysis against the ip of server B (but server C could try to look for an originating IP in the message Received headers. Mox cannot currently do that, and I am not sure the IP can be retrieved reliably). This too implies server C needs to know messages through B are forwarded messages, to adjust its junk analysis. C should probably also use the results in the arc header added by B for its analysis (mox currently cannot parse/sign/verify arc headers).

If C rejects a message from B, what is B supposed to do? It should normally send a dsn back to A, indicating failure to deliver. This is a problem if A is actually a forged address. The dsn of a junk message would go to an innocent address (backscatter). If A was spf-validated, the dsn could be sent without problem. Perhaps we could just not return a dsn to A if its domain wasn't verified? We don't want to drop messages entirely, so we would store it at B for user C to inspect periodically (like a spam box). But now user C has to look at messages stored in B once in a while. That's not good enough, user C might as well keep the account at B and not bother with forwarding at all.

I think the above shows that forwarding to C can only work if C is configured to know about incoming forwarding from B. Otherwise it'll make the wrong decisions (dmarc, spf, ip-based). How is this normally configured in various software operating as C? (Cloud mail services and self-hosted smtp servers, I've only run an unsophisticated postfix installation without dmarc and without account-based junk filtering.)

I think I've arrived at the following conclusion before, but then discarded it for reasons I cannot recall at the moment, so take don't take this for certain: B shouldn't queue-and-deliver messages for C, but only attempt immediate online forwarding between A and C, passing back success or permanent/temporary reject from C to A.

More details/questions:

• After smtp RCPT TO from A, B would dial the forward-destination server, send MAIL FROM, RCPT TO, then copy data between them. A will just see whatever C decides (e.g. accept or temporary/permanent reject). Would B need to translate error codes coming from C? B may not want to respond in a way (with some error codes) it otherwise wouldn't. B would need to take care of some rate limiting towards C, to prevent it from being seen as a bad actor. B would still temporarily store the message while forwarding because it may have to attempt delivery to multiple MX targets. So this isn't a matter of a single forwarded TCP connection. In case of multiple MX targets, if any before the last return a temporary error, the next MX target must be tried. For success or permanent failure, the attempts stop there. In case of temporary errors, it will be on A to retry.
• For transactions with multiple recipients: B should only allow the first recipient to be a forwarding address. For more RCPT TO's, B would refuse the additional recipient and assume A will try again for the additional addresses in a new smtp transaction. Rationale: With an smtp transaction with multiple recipients, we would have to send a dsn if some recipients can and some cannot be delivered to. By refusing multiple recipients during RCPT TO, we never have to send a dsn (which could cause unwanted backscatter). Mox already refuses multiple recipients when spf doesn't pass to prevent having to send dsns.
• Which headers whould B add to the message? Received, Received-SPF, Authentication-Results, arc-headers. Probably Delivered-To? More loop-detecting headers? Probably not a DKIM header? If dkim needs to be added, B needs to first read the full incoming message from A (smtp DATA), calculate the signature, then do DATA to C with the new dkim-signature header(s) prepended. Would the whole message need to be read for ARC as well?
• Which smtp MAIL FROM address should B use? A, B or a rewritten address with B's domain and A's address encoded? Probably just B's address, it's the only one guaranteed to be available.
• So far I've assumed an address is just unconditionally forwarded to another external address. But we'll have Sieve support at some point. Its scripts can cause a forwarding to be initiated. Possibly multiple in a single script, and possibly for other events than a new delivery. Should those also be handled online, without queue-and-deliver, or will different rules apply? The difference could be that such accounts do have messages and (non)junk classifications available, and unsafe dsns could be stored in the account.
• Will this get one of A, B, C into trouble with timeouts? Perhaps if B needs to dial multiple of C's MX targets. Especially when C is applying greylisting (though that should only happen for the first message, or periodically but with long intervals). Perhaps even more so when there are multiple forwardings in play, e.g. A sending to B0, sending to B1, sending to B2, sending to C. Not a great idea to configure forwards like this though.

To come back to your questions:

• Should forwarded email be subject to the same spam checks and whatnot?

I don't think it is feasible for B to do that. But C should do spam checks, but in a special "this is forwarded email"-mode, ideally partially based on headers added by B (that mox wouldn't add currently).

• Should the forwarded emails be retained somewhere in the store?

I hope we can take the forward-online approach instead of queue-and-deliver, and not store the forwarded emails. I wonder what the use case is for storing messages and forwarding them. I know gmail can be configured to do so.

• How much of this sending logic should be shared between submit and this new forwarding function? Or can forwarding actually use submit directly?

Submit puts messages in the queue for delivery. That would be the queue-and-deliver approach. Online forwarding would be a new code path in deliver: Instead of adding the message to the local account, it would dial MX targets of C's domain and attempt to deliver.

Hope this is understandable, and that it makes sense. I may be forgetting situations where this approach would cause problems. There are likely some interactions with (security/authentication) mechanisms that I haven't accounted for. I'm interested in hearing your line of thought and if you've run into anything yet.

[mattfbacon]

only some hop headers [...] possibly dkim.

Use address B. This would cause C to evaluate spf to "pass", but may cause dmarc to evaluate to "fail" at server C because the "message From" address has a different domain than B (addresses are not aligned). This would happen when there is no valid aligned dkim signature to cause a dmarc "pass".

Seems like the solution is to re-sign with DKIM after modifying the From.

It also would only be helpful if we get (non)junk classifications done at C back to B, but there is no clear path to do that.

I don't see this as particularly important. Why not allow C to handle junk classification and skip it on C?

accept all such forwarded messages based on the assumption that B has done junk filtering

Seems fragile and possibly exploitable.

C will get all the junk and can never push back on unwanted senders

Maybe I'm missing something. How would B be able to "push back" any more than C?

If C is running other software, or user C has not configured messages from B as being forwarded messages, junk filtering at C would apply

Are you saying that B somehow needs to know if C is mox or not? I think it's much simpler to just let C handle junk detection.

Since dmarc could no longer "pass", a non-junk message may now be rejected.

Ambiguous English. By "could" do you mean "there is a possibility it will no longer pass" or "it would no longer be possible for it to pass"?

Likewise, server C shouldn't do ip-based junk analysis against the ip of server B

My impression is that B would have to handle this. Everything except Bayesian content analysis would still happen on B, because the same implementation of these policies such as SPF and DMARC would return the same result on both B and C, so there's no reason to not do it on B and save C the work.

This is a problem if A is actually a forged address. The dsn of a junk message would go to an innocent address (backscatter). If A was spf-validated, the dsn could be sent without problem. Perhaps we could just not return a dsn to A if its domain wasn't verified?

Can you clarify what you mean by "forged address"? Don't the normal rules for receiving email apply here? That is, if B is not forwarding and rejects the email, what are the rules for sending a DSN in that case?

I think the above shows that forwarding to C can only work if C is configured to know about incoming forwarding from B. Otherwise it'll make the wrong decisions (dmarc, spf, ip-based).

Please confirm if this still applies with my proposed choices.

B shouldn't queue-and-deliver messages for C, but only attempt immediate online forwarding

Interesting idea, but doesn't this violate the SMTP spec which mandates resend attempts and whatnot?

B would still temporarily store the message while forwarding because it may have to attempt delivery to multiple MX targets. So this isn't a matter of a single forwarded TCP connection. In case of multiple MX targets, if any before the last return a temporary error, the next MX target must be tried. For success or permanent failure, the attempts stop there. In case of temporary errors, it will be on A to retry.

This sounds a lot like queuing and delivering to me...

For transactions with multiple recipients: B should only allow the first recipient to be a forwarding address. [...] Mox already refuses multiple recipients when spf doesn't pass to prevent having to send dsns.

Doesn't that current behavior still work in the case of forwarding? What is the need for the special case? And is this addressed by my previous backscatter comment?

I think the rest of the points in that section only apply if we are special-casing delivery of forwarded messages. Queue-and-deliver already handles these cases AFAICT.

[mjl-]

Seems like the solution is to re-sign with DKIM after modifying the From.

Interesting thought. Rewriting the From message header is done in mailing list software. You'll sometimes see it applied selectively (e.g. google groups), possibly based on spf/dmarc policy of the sender domain. But I don't think this is common behaviour for forwarded messages. And it certainly isn't great to not see the original From header. You want the original From address visible in mail clients, but they only display a limited set of message headers. The Reply-To header could be set/overwritten with the original From address...

It also would only be helpful if we get (non)junk classifications done at C back to B, but there is no clear path to do that.

I don't see this as particularly important. Why not allow C to handle junk classification and skip it on C?

The conclusion (after having dumped brain in the text) does arrive at doing junk classification at C only. I'm assuming you meant to say "and skip it on B"?

accept all such forwarded messages based on the assumption that B has done junk filtering

Seems fragile and possibly exploitable.

Well, if B was able to do junk filtering, that would be great. The sooner in the delivery process the better. But I don't see how it can be done properly at B.

Some additional background: I believe many cloud mail providers will accept spam, and put it in the spam folder (or just drop it altogether!). I think that's not desirable behaviour because legitimate mail gets lost that way in practice. Mox aims only accepts messages if they can be delivered to a regular mailbox. There is a reason the cloud mail providers accept (and hide) instead of reject spam: they don't want to give the spammers any signal about the result of junk analysis. The theory is that they may use that signal to probe the junk filter, to find a way to get past it. It could work in theory, but I don't know if spammers would do this in practice. I tend to think not, given the (lack of) sophistication I tend to see in junk messages. Mox does a temporary reject that spammers can never be sure is the result of junk analysis vs a legitimate temporary server error (it also ties up resources of the spammer).

It would be interesting to learn what other mail servers currently do for these forwarding situations.

C will get all the junk and can never push back on unwanted senders

Maybe I'm missing something. How would B be able to "push back" any more than C?

If B knows (from analysis) that A is a spammer, it can just reject the message until A gives up (A stays responsible for retrying). If B accepts the message from A, and tries to deliver to C, and C rejects the message from B, then A has seen success and B must keep retrying (it accepted the message so is now responsible for delivery). Eventually, B must send back a delivery failure notification to address A. B doesn't learn why C (which has all the info needed for analysis) rejected the message. Does it contain bad content? Is the original sender address/domain/spf/dkim/ip(class) bad? The next time A sends a message, B will accept it again, try delivery to C, which will likely reject again. If sender A is a spammer using a forged address, it will just see its messages get accepted by B. It never has to retry. The DSNs go to a forged address, spammer A won't seen them. So A will keep spamming. B and C will be doing quite some work, and will bother innocent forged address A with a DSN.

If B would know not to accept the messages from A, then A will stay responsible for delivery attempts. B will never have to send a DSN (to an innocent forged address). And spammer A may eventually realize that its delivery attempts are not worth it.

If C is running other software, or user C has not configured messages from B as being forwarded messages, junk filtering at C would apply

Are you saying that B somehow needs to know if C is mox or not? I think it's much simpler to just let C handle junk detection.

I meant to convey that a mox account at C can configure addresses that are known forwarding sources (B), and that mox will adjust its junk analysis for such messages (not rejecting due to DMARC, not doing classification-based IP reputation analysis). I don't know if other mail servers have an option like this. If not, a server C would likely do regular dmarc/junk checks, which could have bad consequences (reject due to dmarc fail, reject due to bad ip reputation of B).

Since dmarc could no longer "pass", a non-junk message may now be rejected.

Ambiguous English. By "could" do you mean "there is a possibility it will no longer pass" or "it would no longer be possible for it to pass"?

Ah yes, I meant "there is a possibility it will no longer pass". I think I should've said "Since dmarc may no longer pass?". If the dmarc pass depended on SPF, because of absence of DKIM signature, with a now-failing aligned-SPF check, the dmarc check would now fail instead of pass.

Likewise, server C shouldn't do ip-based junk analysis against the ip of server B

My impression is that B would have to handle this. Everything except Bayesian content analysis would still happen on B, because the same implementation of these policies such as SPF and DMARC would return the same result on both B and C, so there's no reason to not do it on B and save C the work.

Good point. B should be rejecting based on DMARC (based on SPF and DKIM checks) if it can, without bothering C. But since B won't have (non)junk message classifications, it cannot do most of the other checks: sender address/domain/ip-based or content-based (bayesian). Mox also takes (non)junk-classified verified DKIM and SPF domains into account for filtering. So that's different from using DKIM/SPF for DMARC analysis. This mechanism allows messages from certain ESP (email service providers, e.g. mailchimp), that typically add their own DKIM signature and will be sending from their own IPs (SPF), to be accepted/rejected, regardless of being a first-time sender (without its own reputation).

But C if decides a message is spam, it shouldn't hold it against (the IPs of) server B, like it would normally do for spammy senders. Because it may prevent also legitimate messages coming in through B to be rejected. That's why C needs to know B is just the messenger, not the responsible sender. A default mox configuration uses sending-server-IP-based filtering based on (non)junk classifications of messages.

This is a problem if A is actually a forged address. The dsn of a junk message would go to an innocent address (backscatter). If A was spf-validated, the dsn could be sent without problem. Perhaps we could just not return a dsn to A if its domain wasn't verified?

Can you clarify what you mean by "forged address"? Don't the normal rules for receiving email apply here? That is, if B is not forwarding and rejects the email, what are the rules for sending a DSN in that case?

If B doesn't accept the message during SMTP, the responsibility for delivery attempts stays at A. B does not have (and should not) send a DSN to A. But when B accepts the message from A, it is responsible for either delivery or sending back a DSN if it cannot be delivered. So, as B, I'm trying to avoid accepting messages from A if I don't know that C will accept them. If A is verified, it would be okay, because the address isn't forged and sending the DSN is the right action (though still not great, because spammers send form verified addresses but may not actually accept incoming email causing the queue to fill up with dsns). But if A isn't verified, B doesn't want to be in a position to have to send a DSN to A. For forwarding, we'll probably have to read the full message first, and could do a queue-and-deliver if A is verified (including sending DSN if needed), but do online-forward if A is not verified. A could perhaps be verified with either SPF or DKIM, that would take some more thought/analysis. By the way, the DSN will go to the SMTP MAIL FROM address, which can be different from the "message From" address. I didn't have that distinction in mind when writing my text yesterday, but don't think it would change the conclusion.

I think the above shows that forwarding to C can only work if C is configured to know about incoming forwarding from B. Otherwise it'll make the wrong decisions (dmarc, spf, ip-based).

Please confirm if this still applies with my proposed choices.

I still think C needs to either be aware of incoming forwarding, or just not be sophisticated enough to do DMARC or sending-server-reputation(IP)-based analysis (whether message-classification-based or not).

B shouldn't queue-and-deliver messages for C, but only attempt immediate online forwarding

Interesting idea, but doesn't this violate the SMTP spec which mandates resend attempts and whatnot?

As long as B doesn't accept a message, A will stay responsible for retrying. Once B accepts it, it has to put it in its queue and retry delivery on failure.

B would still temporarily store the message while forwarding because it may have to attempt delivery to multiple MX targets. So this isn't a matter of a single forwarded TCP connection. In case of multiple MX targets, if any before the last return a temporary error, the next MX target must be tried. For success or permanent failure, the attempts stop there. In case of temporary errors, it will be on A to retry.

This sounds a lot like queuing and delivering to me...

Perhaps the terms should be "accept-and-deliver" instead of "queue-and-deliver". The big difference is that with online-forwarding B would only "accept" a message when C has accepted it, and otherwise pass on the rejects (or synthesize a temporary reject if C's servers cannot be reached). So the code/logic at B doing the actual delivery attempt to C would be pretty much the same for both accept-and-deliver and online-forwarding, but with accept-and-deliver B would accept first (after which A is done with the message), then start attempting to deliver.

I think we could run into trouble with large messages combined with slow connections (either A→B or B→C being slow). The connection may time out. The impact could be mitigated by using accept-and-deliver if sender A is verified (where a DSN would be less of a problem). But ideally only when an earlier delivery attempt from B to C has actually failed due to a timeout (this may require keeping track of message-id's of recent forwarding attempts).

For transactions with multiple recipients: B should only allow the first recipient to be a forwarding address. [...] Mox already refuses multiple recipients when spf doesn't pass to prevent having to send dsns.

Doesn't that current behavior still work in the case of forwarding? What is the need for the special case? And is this addressed by my previous backscatter comment?

I think the rest of the points in that section only apply if we are special-casing delivery of forwarded messages. Queue-and-deliver already handles these cases AFAICT.

If we wouldn't change the behaviour for recipients that will be forwarded, and sender A was verified, server B would allow multiple forwarding recipients (B0 and B1) in an SMTP transaction. That would be a problem for online-forwarding, because forwarding B0 to C0 may have a different result than forwarding B1 to C1. What would the response to A be? We can't tell it to retry only one of the recipients, and we don't want it to retry both recipients because it would likely result in messages being delivered multiple times. If the first recipient is a forwarded address, we cannot allow a second recipient at all (even if local) for the same reasons. So the rule would be: B should only allow the first recipient to be a forwarding address, and if so not accept more recipients.

[mattfbacon]

Hm, I see your points and I agree that it's problematic that B becomes responsible for retrying sends to C. So maybe online forwarding is indeed the best way to do it. It would retain A's responsibility to retry which is good because B->C could spuriously fail for whatever reason.

I did a bit more research about how SMTP forwarding is typically done and found https://en.wikipedia.org/wiki/Sender_Rewriting_Scheme. An excerpt:

Receivers can arrange their forwarding in a way that works with SPF with in essence three strategies:

1. not checking SPF behind their border, e.g. white list forwarders
2. just reject SPF FAIL, resulting in a bounce (SMTP error 550)
3. rewrite the MAIL FROM at the forwarder (as done by mailing lists)

Sender Rewriting Scheme (SRS) is one way for the third strategy.

This may be useful here to avoid the issues with SPF we were discussing. I have forwarding set up from my old gmail account and I can see in the headers that it appears to use this scheme.

Also, I'm not sure if there is confusion in the term "From" between the "envelope sender" in the SMTP MAIL FROM command and the "FROM header" inside the email. We are only talking about rewriting the envelope sender, right?

I believe many cloud mail providers will accept spam, and put it in the spam folder (or just drop it altogether!). I think that's not desirable behaviour because legitimate mail gets lost that way in practice.

Yes, this has happened to me and it's very annoying.

I still think C needs to either be aware of incoming forwarding, or just not be sophisticated enough to do DMARC or sending-server-reputation(IP)-based analysis (whether message-classification-based or not).

So the way I understand this is that anything to do with IP reputation and that kind of thing for A needs to be checked at B because C will only be checking for B. And if we are doing online forwarding then a rejection from B and a rejection from C can look essentially the same to A.

If we wouldn't change the behaviour for recipients that will be forwarded, and sender A was verified, server B would allow multiple forwarding recipients (B0 and B1) in an SMTP transaction. [...] We can't tell it to retry only one of the recipients, and we don't want it to retry both recipients because it would likely result in messages being delivered multiple times.

I see the trouble here. So by rejecting the email we are hoping that the sending MTA will retry with each address separately? Is this in the SMTP spec?

[mattfbacon]

Oh also I want to add that Outlook does forwarding wrong and messes up DKIM. So we can use it as a reference of how not to do forwarding :)

[mjl-]

I did a bit more research about how SMTP forwarding is typically done

Another relevant resource is https://datatracker.ietf.org/doc/html/rfc7960, about issues with dmarc/spf/dkim, including but not limited to forwarding.

Also, I'm not sure if there is confusion in the term "From" between the "envelope sender" in the SMTP MAIL FROM command and the "FROM header" inside the email. We are only talking about rewriting the envelope sender, right?

Yes. Other terms used are "rfc5321.mailfrom" for "envelope/smtp mail from" and "rfc5322.from" for "message from".

So the way I understand this is that anything to do with IP reputation and that kind of thing for A needs to be checked at B because C will only be checking for B. And if we are doing online forwarding then a rejection from B and a rejection from C can look essentially the same to A.

Correct. And so actually C should be configured in a way that doesn't do IP checking against B.

I see the trouble here. So by rejecting the email we are hoping that the sending MTA will retry with each address separately? Is this in the SMTP spec?

Exactly, and the SMTP spec does have a section about a recipient limit, and error that should be sent back, and client behaviour for sending to the remaining recipients in a new transaction, but the approach would be in violation of the spec because we must accept at least 100 recipients. Mox is already doing this (in violation), and I would think this is a common technique to prevent backscatter, but am not sure.

https://github.com/mjl-/mox/blob/8a866a60dcba1e9e008f177eca076371d06a015f/smtpserver/server.go#L1414 https://datatracker.ietf.org/doc/html/rfc5321#section-4.5.3.1.8 https://datatracker.ietf.org/doc/html/rfc5321#section-4.5.3.1.10 https://datatracker.ietf.org/doc/html/rfc3463#section-3.6, see X.5.3 Too many recipients.

The limit of 100 recipients is also still in a work-in-progress revision of SMTP, https://datatracker.ietf.org/doc/draft-ietf-emailcore-rfc5321bis/

It would be good to check if other implementations do this as well, and whether they handle a recipient limit of 1 gracefully.

Oh also I want to add that Outlook does forwarding wrong and messes up DKIM. So we can use it as a reference of how not to do forwarding :)

What does it do? Modify the message so the DKIM-signature breaks?

Another thing to consider is this forwarding scenario: A→B→C→D. If B rewrites the "envelope mail from" when sending from A to C, and C accepts-and-delivers the message, but delivery to D fails, then C will be bouncing to B instead of directly to A, to the address B rewrote. And B would still have to send a DSN to A, which we were trying to avoid. This is an argument to not apply sender rewriting at B. C needs modifications for its DMARC handling anyway, so C not needing an SPF pass doesn't seem too far fetched.

[mattfbacon]

And so actually C should be configured in a way that doesn't do IP checking against B.

Maybe, but would there be an issue with C checking B anyway?

we must accept at least 100 recipients. Mox is already doing this (in violation)

Really? I thought the MTA could reject emails for any reason. e.g. "Temporary failure"

What does it do? Modify the message so the DKIM-signature breaks?

I'm not entirely sure but their messages always get rejected due to DKIM and it says "body hash did not verify".

[mattfbacon]

Another thing to consider is this forwarding scenario: A→B→C→D. [...] This is an argument to not apply sender rewriting at B.

To me it seems more like an argument to not accept-and-deliver.

[mjl-]

Maybe, but would there be an issue with C checking B anyway?

Not if C wouldn't start blocking. But if there are a lot of bad A's trying to send spam, then with some IP-based logic C may start considering B is a spammer too. Rate-limiting is another topic that C may have to adjust if B is just the messenger.

Really? I thought the MTA could reject emails for any reason. e.g. "Temporary failure"

You can reject, and the sender has to retry, but then nothing was sent. We do want the sender to make progress with 1 recipient. So we want them to continue delivering after having seen "too many recipients"

I'm not entirely sure but their messages always get rejected due to DKIM and it says "body hash did not verify".

Yeah, that sounds like the message got modified...

To me it seems more like an argument to not accept-and-deliver.

True, but C may not be under our control. Users may be setting up all kinds of forwarding scenario's. Though perhaps the user will unconfigure forwarding chains when they start giving problems...

[mjl-]

Also interesting, Forward Pass: On the Security Implications of Email Forwarding Mechanism and Policy: https://www.sysnet.ucsd.edu/~voelker/pubs/forwarding-eurosp23.pdf

[mattfbacon]

Also interesting, Forward Pass: On the Security Implications of Email Forwarding Mechanism and Policy: https://www.sysnet.ucsd.edu/~voelker/pubs/forwarding-eurosp23.pdf

So if I understand correctly, we would be implementing the REM approach?

[mjl-]

I didn't come to any conclusions yet from reading that. I still think there is no way around requiring configuration of a forward at the recipient mail server (to adjust its dmarc/ip filtering). Forwarding mail servers (intermediates) seem to have different behaviour. Perhaps we'll to provide config options where a user can configure the kind of forwarding that is happening?

In the (far) future, perhaps a standardized mechanism could be designed for mail servers to set up forwarding. Some mail servers require verified opt-in before forwarding (probably with an email with a verification link). Some structured fields in such a message could do the trick. But that doesn't exist now.