search

deltachat / deltachat-core-rust

Delta Chat Rust Core library, used by Android/iOS/desktop apps, bindings and bots 📧
https://delta.chat/en/contribute
Other
669 stars 85 forks source link

Messages sent to invalid mail addresses are marked as "Delivered" in "Message details" #1478

Closed gerryfrancis closed 4 years ago

gerryfrancis commented 4 years ago
Jikstra commented 4 years ago

This is not really easy to implement, because we actually send this message. No matter if the address is valid or not, if we send it out via SMTP it's getting sent and we therefore also think this was a valid sent message. But somehow parsing those Undelivered Mail reports would be quite neat, but of course a core feature and not something we can do on the desktop side of things.

gerryfrancis commented 4 years ago

@Jikstra I agree, this would not be an easy task to solve. Nevertheless, the term "Delivered" only applies for messages that really arrived on the other side. I guess it would be better if "Delivered" is changed to "Sent", otherwise a user of Delta Chat would think that the message sent was indeed delivered.

Jikstra commented 4 years ago

Yup I agree, sent makes more sense. The only way that we know the other side received it, is if we get a read notification.

gerryfrancis commented 4 years ago

The only way that we know the other side received it, is if we get a read notification.

@Jikstra As matters stand, you are right.

Simon-Laux commented 4 years ago

The content in the grey box inside of the message info dialog comes from the core.

gerryfrancis commented 4 years ago

@Simon-Laux Thanks for moving to the right place. :+1:

Hocuri commented 4 years ago

@r10s Do you think parsing "delivery failed" emails would be worth the effort? (shall I implement this?)

Simon-Laux commented 4 years ago

Is there an universal way to detect undelivered emails or do we need to have multiple parsers one for each provider or better one for each mail software the providers can use? Looking at the email source code of the undelivered mails of two different providers didn't reveal an obvious universal way for me.

I like the feature, but unless the work is not that much, there might be more important things to go for first. Though I believe we should somehow notify the user of undelivered-email-warnings of the providers mailer daemon if they are related to chat-emails, even though they aren't chat messages, they should be displayed somewhere in the app.

Hocuri commented 4 years ago

Well, if we find out that it is an undelivered-email-warning, we can also set the delivery status to failed because the Message-ID of the failed message is in In-Reply-To (I hope that this is always the case, maybe you can tell me?) (and I always hope that we somehow store the original Message-ID of the emails, because this might get tricky otherwise).

Just noticed: Heuristics are bad because anyone could send fake undelivered-email-warnings to set the message state to failed. We would need an extra-entry MailDeliveryDaemon in the providers database. In this case we could at least be sure that this is a undelivered-email-warning and the message in In-Reply-To should be marked as failed.

Simon-Laux commented 4 years ago

unfortunately the two german email providers I just looked at don't set the in-reply-to header in their mail-delivery-failed messages.

Simon-Laux commented 4 years ago

We would need an extra-entry MailDeliveryDaemon in the providers

That would give us a way to directly whitelist the chat with it so the user doesn't misses mail delivery failed messages.

hpk42 commented 4 years ago

In many cases it is possible i think to know which message-sending failed, even if a mailer-daemon is not "in-reply-to" the one that went out. Usually the original message (headers) are included or at least the affected Message-ID and you have to fish it out.

I think improved user-visible error-handling is totally worth the effort because it relates to the very basic operation of a messsenger: sending and receiving messages.

And this includes giving feedback on "this message has and will never see an addressed user/device". If we do mark a message as "failed" then there needs to be a way to find out why it failed (probably in message-info/more). And note that in group chats only a fraction of recipients may fail.

On Tue, May 19, 2020 at 08:59 -0700, Simon Laux wrote:

We would need an extra-entry MailDeliveryDaemon in the providers

That would give us a way to directly whitelist the chat with it so the user doesn't misses mail delivery failed messages.

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/deltachat/deltachat-core-rust/issues/1478#issuecomment-630915717

Hocuri commented 4 years ago

I'll call undelivered-email-warnings NDN (Non-delivery-notifcation).

What about this:

--

--

Wikipedia: https://en.wikipedia.org/wiki/Bounce_message#Format Mime-parts in NDNs from Google: 

--00000000000012d63005a5ee9520
Content-Type: message/delivery-status

Reporting-MTA: dns; googlemail.com
Arrival-Date: Mon, 18 May 2020 09:23:25 -0700 (PDT)
X-Original-Message-ID: <CABXKi8zruXJc_6e4Dr087H5wE7sLp+u250o0N2q5DdjF_r-8wg@mail.gmail.com>

Final-Recipient: rfc822; assidhfaaspocwaeofi@gmail.com
Action: failed
Status: 5.1.1
Diagnostic-Code: smtp; 550-5.1.1 The email account that you tried to reach does not exist. Please try
 550-5.1.1 double-checking the recipient's email address for typos or
 550-5.1.1 unnecessary spaces. Learn more at
 550 5.1.1  https://support.google.com/mail/?p=NoSuchUser i18sor6261697wrs.38 - gsmtp
Last-Attempt-Date: Mon, 18 May 2020 09:23:25 -0700 (PDT)

[...]

--00000000000012d63005a5ee9520
Content-Type: message/rfc822

DKIM-Signature: …
X-Google-DKIM-Signature: …
X-Gm-Message-State: AOAM531vhwpXiK8M12286dOJx0Q5fBl9ZaH6BJKts93GoxvPv0xdryP0
    jg9wYmoP5MUHudsxAMCYDFsCUMVx2PEywyIsaQqklw==
X-Google-Smtp-Source: …
X-Received: by 2002:a5d:5183:: with SMTP id k3mr20545162wrv.159.1589819005034;
 Mon, 18 May 2020 09:23:25 -0700 (PDT)
MIME-Version: 1.0
From: …
Date: …
Message-ID: <CABXKi8zruXJc_6e4Dr087H5wE7sLp+u250o0N2q5DdjF_r-8wg@mail.gmail.com>
Subject: Kommt sowieso nicht an
To: assidhfaaspocwaeofi@gmail.com
Content-Type: multipart/alternative; boundary="0000000000000d652a05a5ee95df"
hpk42 commented 4 years ago

On Wed, May 20, 2020 at 06:02 -0700, Hocuri wrote:

I'll call undelivered-email-warnings NDN (Non-delivery-notifcation).

What about this:

  • Recognize NDNs by the sender address, which is in the providers database. The problem with heuristics would be that someone could send fake-UEWs and mark messages as failed.
  • Look for In-Reply-To
  • Look for a mimepart message/delivery-status, which might contain a X-Original-Message-ID (see below).
  • Search through the body and look for a line Message-ID: * (remember to remove <, >), in the hope that that's the original header
  • Look for a mime-part message/rfc822, which will contain the original header. In there, look for a line Message-ID: *.

matches what i remember about this topic. I think it'd be good to have a little test corpus of example mails in a data-directory, and then some rust tests that call dc_receive_imf and check that the failed-deliver reflects in the message status and msg_info.

I also think that this work does not interfere much with async-core PR which is helpful ;)