search

deltachat / deltachat-core-rust

Delta Chat Rust Core library, used by Android/iOS/desktop apps, bindings and bots 📧
https://delta.chat/en/contribute
Other
664 stars 85 forks source link

Disappearing messages if "Never" is chosen in a dialogue #2346

Closed dumblob closed 3 years ago

dumblob commented 3 years ago

This issue I observe from the very first time I tried DC (which is about a year back). It arises if one uses one email account for chat over DC (with just 10 or 20 chosen email addresses) along with a non-Autocrypt-enabled MUA for email communication.

I'm also using the Not now button and it works perfectly fine.

I'd suggest not to delete the email but leave it as it is (basically let the Never button do the same as Not now with the only difference of adding the email sender to the black list). If that's not desired (for whatever reason I'm currently unable to see or justify), then I'd like the email moved to the Trash special folder (as RFC suggests) and if the Trash folder is not available then try Archive and if it also doesn't exist ask user where to move it.

But please never delete it (I've lost already about 5 important emails which caused me more work and headaches due to not realizing it in the first place because the information about "missing something" doesn't exist anywhere - I actually found out by accident when reading my persisted history of SMS notifications which contain also notifications about certain new emails from my email account).

gerryfrancis commented 3 years ago

@dumblob Thank you for your report.

the email is unrecoverably deleted (the email sender is added to the black list as expected)

I followed your steps carefully but my result is different: After I blocked the new contact, the message (mail) remains untouched in my INBOX folder. (I checked that with my Webmail client.) What happens when you unblock the contact? Do the messages appear in a chat view?

adbenitez commented 3 years ago

I like messages from blocked contacts are deleted immediately, blocking is not the same as what you seem to want "ignore emails from this contact so I can read them in classic email app", maybe instead of "never" it should say clearly "block"?

Edit by @Hocuri, I hope that's OK for you, adb: Note that, even though this would be useful to people who want it, this is not the expected behavior. We do not deliberately delete messages without explicitly asking the user. And blocking does mean what @dumblob expected. (@adbenitez and I are both admins here, so that we can edit each other's messages)

dumblob commented 3 years ago

I followed your steps carefully but my result is different: After I blocked the new contact, the message (mail) remains untouched in my INBOX folder. (I checked that with my Webmail client.) What happens when you unblock the contact? Do the messages appear in a chat view?

Thanks @gerryfrancis . I've tried with the two contacts I remember losing a message from them. And no, they don't appear anywhere - I get a new chat with No messages. subtitle (and after clicking on this chat there are no messages and instead the welcoming info buble).

Did you turn on/off all the settings I outlined in your desktop client?

I like messages from blocked contacts are deleted immediately, blocking is not the same as what you seem to want "ignore emails from this contact so I can read them in classic email app", maybe instead of "never" it should say clearly "block"?

Then it should say "delete and block" and not delete the message but move it to Trash/Archive (where it'll be deleted after some expiration time automatically - that's I suppose what you're expecting in case DC is your only mail client to avoid emails from blocked contacts piling up). The point is, that at that point it's not yet considered by the user "under the umbrella of DC" so deleting it is really bad user experience.

Despite this possibility I disagree with anything but leaving the message intact in INBOX and claim that it's a bad user experience to do something more than blocking. DC is designed to coexist with standard email workflow and enormous amount of effort is being invested into this coexistence to be smooth. Such deleting goes directly against this goal.

Either way we need to sort out why it behaves in my and @adbenitez's case differently than in @gerryfrancis's case. It feels really fishy :cry:.

dumblob commented 3 years ago

From the logs this is what happened in the morning yesterday:

2021-04-14T06:56:51.707Z    core/event              INFO        "DC_EVENT_INFO" 0   "src/imap/idle.rs:71: INBOX: Idle entering wait-on-remote state"
2021-04-14T06:58:51.961Z    core/event              INFO        "DC_EVENT_INFO" 0   "src/imap/idle.rs:87: Idle has NewData ResponseData { raw: 4096, response: MailboxData(Exists(15487)) }"
2021-04-14T06:58:52.023Z    core/event              INFO        "DC_EVENT_INFO" 0   "src/job.rs:1270: loading job for Imap-thread"
2021-04-14T06:58:52.163Z    core/event              INFO        "DC_EVENT_INFO" 0   "src/contact.rs:518: added contact id=480 addr=jgznieuwsbrief@ggdtwente.nl"
2021-04-14T06:58:52.390Z    core/event              INFO        "DC_EVENT_INFO" 0   "src/dc_receive_imf.rs:66: Receiving message INBOX/56869, seen=false..."
2021-04-14T06:58:52.430Z    core/event              INFO        "DC_EVENT_INFO" 0   "src/dc_receive_imf.rs:1673: not creating ad-hoc group: too few contacts"
2021-04-14T06:58:52.467Z    core/event              INFO        "DC_EVENT_INFO" 0   "src/dc_receive_imf.rs:1026: Message has 1 parts and is assigned to chat #Chat#355."
2021-04-14T06:58:52.478Z    core/event              INFO        "DC_EVENT_INFO" 0   "src/dc_receive_imf.rs:298: received message 56869 has Message-Id: c1f8646a38a3a5644dc2d9ac760ca42d@eprov.nl"
2021-04-14T06:58:52.488Z    core/event              INFO        "DC_EVENT_INFO" 0   "src/imap/mod.rs:737: 1 mails read from \"INBOX\"."
2021-04-14T06:58:52.553Z    core/event              INFO        "DC_EVENT_INFO" 0   "src/imap/mod.rs:737: 0 mails read from \"INBOX\"."

(I didn't delete nor redact anything from this snippet - it's pure copy&paste of a part of the log)

I've blocked this contact (jgznieuwsbrief@ggdtwente.nl) few hours later, but couldn't find anything related in the logs. Before "blocking" I've checked with regular MUA and the email from jgznieuwsbrief@ggdtwente.nl was in INBOX and unread (just as I would expect). Then immediately after blocking I've checked again and it was gone (I've looked in all email folders incl. DC folder, Trash, Archive etc. and then even under Contact request in DC).

gerryfrancis commented 3 years ago

Did you turn on/off all the settings I outlined in your desktop client?

@dumblob No, I did the test on Delta Chat for Android, shame on me! Development for the Desktop app has fallen back a little bit, but the developers will merge all relevant changes time after time. Could you maybe perform the same test with a current nightly (beta) build of Delta Chat for Android and report back, please?

Hocuri commented 3 years ago

The current expected behavior is definitely not to delete the message, as @dumblob expects it. We can maybe add a "This contact is spam, immediately delete all messages" feature in the future, but we must not delete any messages without telling the user.

Many users will never again use DC if it deleted a message they still needed.

@adbenitez Does this mean that you can reproduce this? Does DC delete emails on the server because you blocked the sender? Don't you have "Delete messages from server" turned on, anyway?

(Quick context for @dumblob: We have a feature called "Delete messages from server" where you can configure DC to immediately delete all incoming messages, if you want this. But there is a very big warning message if you want to enable it, so no risk of a user turning it on by accident. As opposed to just hitting Never.)

Hocuri commented 3 years ago

As someone who knows the core code well, my first reaction would even be "it's impossible, you must be wrong, there is no way DC deleted these messages". But I've been surprised by what's possible in large codebases.

@dumblob can you provide some more logs? If you click "View -> Developer -> Open the log folder", you get the logs until several weeks back. We are especially looking for the word "delete" (case-insensitive and not only whole words but also e.g. "DeleteMsgOnImap").

dumblob commented 3 years ago

@dumblob can you provide some more logs?

Sure. But because it's my primary email account (yes, I've built quite some trust in DC over the course of the last ~2 years), could I send it to you privately (e.g. through DC)?

Btw. it's weird - cd ~/.config/DeltaChat && grep -Ei delete logs | sort | tail -n 14 gives:

2021-04-11-11-46-41.log:2021-04-12T18:13:28.936Z  core/event              INFO    "DC_EVENT_INFO" 0 "src/job.rs:1125: Inbox removes job #6905, action DeleteMsgOnImap as it succeeded"
2021-04-11-11-46-41.log:2021-04-14T13:59:48.743Z  core/event              INFO    "DC_EVENT_INFO" 0 "src/job.rs:200: saving job for Imap-thread: Job { job_id: 0, action: DeleteMsgOnImap, foreign_id: 25620, desired_timestamp: 1618408788, added_timestamp: 1618408788, tries: 0, param: Params { inner: {} }, pending_error: None }"
2021-04-11-11-46-41.log:2021-04-14T13:59:48.777Z  core/event              INFO    "DC_EVENT_INFO" 0 "src/job.rs:1075: Inbox-job #6909, action DeleteMsgOnImap started..."
2021-04-11-11-46-41.log:2021-04-14T13:59:48.777Z  core/event              INFO    "DC_EVENT_INFO" 0 "src/job.rs:1144: Inbox begin immediate try 0 of job #6909, action DeleteMsgOnImap"
2021-04-11-11-46-41.log:2021-04-14T13:59:48.778Z  core/event              INFO    "DC_EVENT_INFO" 0 "src/job.rs:584: Running delete job for message c1f8646a38a3a5644dc2d9ac760ca42d@eprov.nl which has 1 entries in the database"
2021-04-11-11-46-41.log:2021-04-14T13:59:48.921Z  core/event              INFO    "DC_EVENT_INFO" 0 "src/job.rs:1168: Finished immediate try 0 of job #6909, action DeleteMsgOnImap"
2021-04-11-11-46-41.log:2021-04-14T13:59:48.922Z  core/event              INFO    "DC_EVENT_INFO" 0 "src/job.rs:1125: Inbox removes job #6909, action DeleteMsgOnImap as it succeeded"
2021-04-11-11-46-41.log:2021-04-14T13:59:57.972Z  core/event              INFO    "DC_EVENT_INFO" 0 "src/job.rs:200: saving job for Imap-thread: Job { job_id: 0, action: DeleteMsgOnImap, foreign_id: 25618, desired_timestamp: 1618408797, added_timestamp: 1618408797, tries: 0, param: Params { inner: {} }, pending_error: None }"
2021-04-11-11-46-41.log:2021-04-14T13:59:58.013Z  core/event              INFO    "DC_EVENT_INFO" 0 "src/job.rs:1075: Inbox-job #6911, action DeleteMsgOnImap started..."
2021-04-11-11-46-41.log:2021-04-14T13:59:58.013Z  core/event              INFO    "DC_EVENT_INFO" 0 "src/job.rs:1144: Inbox begin immediate try 0 of job #6911, action DeleteMsgOnImap"
2021-04-11-11-46-41.log:2021-04-14T13:59:58.013Z  core/event              INFO    "DC_EVENT_INFO" 0 "src/job.rs:584: Running delete job for message mCAvwSIxn.CCB5FC2065tiE3.g2IMWJ.4e7233b709.b23b27e7@mail.localhookupmatches.com which has 1 entries in the database"
2021-04-11-11-46-41.log:2021-04-14T13:59:58.120Z  core/event              WARNING   "DC_EVENT_WARNING"  0 "src/imap/mod.rs:1235: Cannot delete on IMAP, spam ko&AWE-/2745: imap entry gone 'mCAvwSIxn.CCB5FC2065tiE3.g2IMWJ.4e7233b709.b23b27e7@mail.localhookupmatches.com'"
2021-04-11-11-46-41.log:2021-04-14T13:59:58.150Z  core/event              INFO    "DC_EVENT_INFO" 0 "src/job.rs:1125: Inbox removes job #6911, action DeleteMsgOnImap as it succeeded"
2021-04-11-11-46-41.log:2021-04-14T13:59:58.150Z  core/event              INFO    "DC_EVENT_INFO" 0 "src/job.rs:1168: Finished immediate try 0 of job #6911, action DeleteMsgOnImap"

But I've clicked Never on 2021-04-14. So maybe I was really too hasty filing the bug on DC core, but it might actually be something in the desktop app. I don't use DC on my phone yet (as @gerryfrancis suggested), but if it's possible already to transfer all data from desktop to phone and since then use both apps (desktop & smartphone) simultaneously while seeing the same content (both received & sent messages, new groups & chats, deleted groups & chats, etc.) synchronized on both devices, then I'll do what you asked for and try it with my phone. Otherwise I'll create a new email account at the same provider a see whether I can reproduce it on a phone.

Hocuri commented 3 years ago

could I send it to you privately?

Sure, it's hocuri1 at testrun.org (don't want to use the "at" character to keep off bots).

And yes, it's possible to transfer all data and then use Desktop and Android simultaneously; it still needs some more love, but generally works already:

adbenitez commented 3 years ago

Actually, the messages are deleted for me because I use "delete messages from server at once" option, it makes sense to avoid deleting important emails by accident and blocking a contact to ignore them in delta chat and handle them in classic email app is also a common use case I have seen in users. So I agree the messages should not be deleted for blocked contacts, and that is probably the current behavior already :thinking:

gerryfrancis commented 3 years ago

the messages should not be deleted for blocked contacts, and that is probably the current behavior already

@adbenitez I cannot confirm this behavior with core version 1.51. (Delta Chat for Desktop still uses version 1.50...)

dumblob commented 3 years ago

FYI I'm in contact with @Hocuri and we're discussing and testing it in more detail. We'll get back once we found something. Probably tomorrow.

gerryfrancis commented 3 years ago

@dumblob Thanks for the info update, I am sure your issue will be in good hands. ;)

Hocuri commented 3 years ago

Probably it's a DC Desktop issue, I opened an issue there: https://github.com/deltachat/deltachat-desktop/issues/2225.

I'll close this issue here; in case it turns out that we do need to change something in the core, let's open a new issue as this one is pretty cluttered with comments already :)