Add a option to delete old messages from the server

[webratte]

If someone use a free mailer like GMX only for Delta Chat he/she never will check if the storage limit is reached.

If messages will deleted after a time (as userpreference) there should be no problems.

[testbird]

@webratte, @csb0730 and others, do you see your use-case covered with the proposed basic implementation plan (auto-delete oldest messages (possibly deleting all but 3 messages if needed, ensuring to stay operational by default)?

[webratte]

For me it sounds like a good solution 👍

[r10s]

after some discussions on #deltachat irc from Jan 09 2019 00:00:01-00:36:00 CEST and ongoing:

• a simple option to delete old messages after a given timesspan from the server would solve a lot of problems

[csb0730]

All the best for new year 2019 to all :-)

and agree to @r10s comment.

This would be easy to handle by local client and gives every user the possibility to customize if required (user setting).

As I read the whole conversation today I saw that restricted server space is an issue. Related to that, time for keeping sent messages at server should be customizable too! An option 'not upload sent messages to imap' could be an alternative.

In general, as I see all these discussions related keeping/deleteing messages:

There seems to be three use cases for mail server:

1. server is only transport backbone for messages (no need to store much data, only to cover not available device(s))
   • incoming msg: can be deleted as soon as downloaded (or as required),
   • sent msg NOT uploaded to imap and NOT bcc to self
2. server is transport backbone and syncs multi devices (store ony for sync)
   • incoming msg: stay at server, deleted (1st client) after time period to guarantee multi dev sync
   • sent msg uploaded to imap or bcc to self (deleted as required)
3. server is main message store and every device is able at every time to sync (much server space needed)
   • incoming msg: stay at server as long as possible,
   • sent msg uploaded to imap or bcc to self

As I see the discussion DC now should cover item 2 (multi device, @r10s position) and @testbird tends to go in direction item 3 (server is main msg location) and @webratte goes for item 1.

To bring all this to a structured discussion and spec, this comment here can be a kick-off point. Up to now a really structured discussion we didn't had.

I think all three use cases have their justification but I think too, that 1. and 2. should be in focus at first. As I see 1. could be implemented very quick in introducing a user setting and simply disable some functions which are now active in current DC versions. Other operation modes are more complicated.

This is enough for now and should be continued and clarified.

[csb0730]

I forgot an option: 'do not delete messages at server at all'. This can be done by server settings too!

[adbenitez]

and @webratte goes for item 1.

just to say that this is my case and the case of all Cuban users, and probably the case of anyone with a paid/low connection, we prefer not to do sync and use an offline alternative like using a backup or other alternative that doesn't involves redownloading the messages from the server

[STPKITT]

Having a local backup would prevent redownloading messages, so it would be no disadvantage in terms of traffic to have all the messages on the server, too. I'd only use item 3 & many people I know too and I think having a solid store of the history is a feature that's very important for any IM.

So while item 1 and 2 of course have their right to exist item 3 should always stay.

[adbenitez]

Having a local backup would prevent redownloading messages, so it would be no disadvantage in terms of traffic to have all the messages on the server, too.

no, will be disadvantage in terms of a waste of time, users can't keep importing/exporting backups, which is annoying and slow, backups are fine for a 1 in a month use, but not to keep things in sync

[adbenitez]

Also, keeping the message on the server is not a desire for users of option 1 because they actually don't want multi-client to work, since downloading the same message in several apps is a waste of time and money for us.

[STPKITT]

In your first comment you said you prefered local backups & now you say backups are a waste of time. Whatever.

[adbenitez]

@STPKITT no, what I meant is that I prefer backups, in the rare case I need to sync two clients, but most of the time I don't even want to sync, what you propose will force people to often import/export backups to avoid the automatic sync operation that drains their money.

[adbenitez]

we are not interested in this sync feature at all, in the rare case we want to sync with other client we just import/export a backup that is what I meant.

[webratte]

Since I use the Android client AND also the desktop client (it's great to write longer messages on PC).

It's very important to me to keep both clients in sync.

[adbenitez]

That is perfectly ok, what is not ok is that because you and some other users like to have 2 client with the same data repeated (and well also repeated in the server), other users that don't want this have to suffer the consequences (this cost money to us also it is slow since connectivity sucks) since sync can't be disabled :/

[webratte]

I'm always a fan of user preferences.

I also would prefer a option to enable/disable sync.

But still I think there are many people who like to use more clients (tablet, phone desktop) to start a chat with one device and continue it on the other.

Don't you think so?

BTW. Just for couriosity....

Is it impossible or forbidden for you to use a mailproviderin europe (e.g. gmx.de or mailbox.org)?

[adbenitez]

But still I think there are many people who like to use more clients (tablet, phone desktop) to start a chat with one device and continue it on the other. Don't you think so?

I am not against sync, it is useful to have this option, but for us this option is REALLY bad, we simply can't sync it will drain our money!!! so please understand: I am not asking to remove sync from DC but just asking for a way to disable it!!!

Is it impossible or forbidden for you to use a mailproviderin europe (e.g. gmx.de or mailbox.org)?

Cuba situation: Internet access is REALLY expensive and not available everywhere, sure I am allowed to use another email provider if, for example, I pay for 30 hours of Internet, but I will not be able to eat that month ;) on the other hand there are an email-only service that is available everywhere (wifi, 2g, etc) and that is much more cheaper than internet.

[STPKITT]

@Ok, I now understand what you meant.

Of course I'm also for giving the user options on how he wants to operate DeltaChat. Nothing worse than programs/apps that want to force users to a certain narrow minded workflow just to keep the option menu as lean as possible which seems to be trendy :-(

[hpk42]

On Wed, Jan 09, 2019 at 18:42 -0800, STPKITT wrote:

@Ok, I now understand what you meant.

Of course I'm also for giving the user options on how he wants to operate DeltaChat. Nothing worse than programs/apps that want to force users to a certain narrow minded workflow just to keep the option menu as lean as possible which seems to be trendy :-(

So FB's privacy settings are a good example for how things should be done? :) /me couldn't resist

[STPKITT]

I never logged into Facebook, but I guess you're refering to how they reportedly make it deliberately difficult for the user like all the companies do that make their money from their user's personal data. Thankfully DeltaChat doesn't belong to that group so the options menu can be designed tidily and logically.

[webratte]

@hpk42

Currently users have to use Deltachat how devs think it's the best way.

For example sync on.

Why should it be bad to add a option to disable sync?

Let it by default on and give the user the chance to switch it off. This way "normal" users can use it the same way as without a option and interested users can change it.

To keep options user friendly you should use categories in the options page. E.g.:

Privacy Send read notification use encryption

Notificatons Selct ringtone Vibration

Design Dark/light design Select background picture

Network *Sync messages between clients

Media Set compress level for pictures Set compress level for videos

To have the choice is always user friendly if you make a user friendly UI.

Facebook is a really stupid example ;-P

[adbenitez]

@webratte thanks for pointing out that "media" category for compression options users here REALLY need it because of the cost and low speed of the network.

Anyway, while our requests are valid, the devs are really busy working on the new release and we are distracting them with this discussion here, we need to be patient and go one step at the time :) perhaps the forum is a better place for discussing about this.

Good night.

On January 11, 2019 12:13:14 AM GMT-05:00, webratte notifications@github.com wrote:

@hpk42>

Currently users have to use Deltachat how devs think it's the best way.>

For example sync on.>

Why should it be bad to add a option to disable sync?> Let it by default on and give the user the chance to switch it off.> This way "normal" users can use it the same way as without a option and interested users can change it.>

To keep options user friendly you should use categories in the options page.>

E.g.:>

Privacy>

• Send read notification>
• use encryption>

Notificatons:>

• Selct ringtone>
• Vibration>

Design>

• Dark/light design>
• Select background picture>

Network>

• Sync messages between clients>

Media>

• Set compress level for pictures>
• Set compress level for videos>

To have the choice is always user friendly if you make a user friendly UI.>

Facebook is a really stupid example 😋>

-- > You are receiving this because you commented.> Reply to this email directly or view it on GitHub:> https://github.com/deltachat/deltachat-core/issues/164#issuecomment-453381005

[webratte]

The only thing I say is:

Keep the design a clear structure (categories) from the beginning, so it's easier to extend it later.

And options are a good thing if you make it in a clear UI.

Not all my examples have to finished. That's only examples 😉

Good morning. 😜

[csb0730]

By the way: adding this user option to disable sync is not as difficult as developping a new function. It's simply to disable existing imap upload and additional fetching of inbox and other folders! Maybe each function independent from the other (upload/download). :-)

[webratte]

Since a few of my friends also use DC it becomes my default Chat App.

So I get more mails then ever.

I think this issue becomes more important then I thought before.

It would be great to have a option to automatically delete older messages (lets say older then 6 month or a user preference) from the server.

Power user will fill up ther mailbox very fast 😉

[adbenitez]

adding a provisional option to manually trigger an action that deletes the DeltaChat folder and recreates it as an easy and faster way to empty the DC folder, will be really appreciated

[csb0730]

Please search for issue "limit retained history". This describes all You mentioned before.

By the way: With gmx there is a setting at web interface to limit storage time of messages for every individual folder of server! I'm using this function too. So You'll not run into trouble in standard use case :-)

[csb0730]

This is the issue I mentioned: https://github.com/deltachat/deltachat-core/issues/244

[webratte]

@csb0730 if you think the idea behind DC is a new toy for nerds your answer is fine.

But I believe the idea behind DC is a new secure AND user friendly messenger. In this case it's never a good idea to force users to search for hidden server settings which eventually exist for some mailproviders. A messenger have to work out of the box.

Users will not search for server settings outside of DC. They will find other messenger apps like Signal or WhatsApp where all can do inside the App.

BTW. my mailprovider (mailbox.org) offers not the option to limit storage time of messages. Should I change my mail provider to use DC?

[hpk42]

On Thu, Feb 14, 2019 at 21:06 -0800, webratte wrote:

@csb0730 if you think the idea behind DC is a new toy for nerds your answer is fine.

This issue is about figuring out how to get rid of old messages stored in the server, right? If some people can modify server settings to achieve that like @csb0730 did, that's useful to point out. Some things are done much more easily at the provider site than with Delta Chat.

There are many different IMAP server configurations that Delta Chat needs to deal with and sometimes fails to deal with. Safely deleting messages from the server which are in DC's local store will take work and testing, as it's a destructive workflow (you might delete the wrong server side mails!). There are also people in at-risk situations who want to delete local chats, but keep the server state. There are UX questions involved on all three apps (android, desktop, ios), apart from API questions for deltachat-core.

Just mentioning this to clarify that what might look like an easy "Let's just introduce an option to do X" is usually embedded in a nexus of considerations, and many other developments and discussions happening. But i am optimistic we eventually implement a way to satisfy the needs expressed in this quite long issue thread :)

holger

[csb0730]

@webratte please do not understand me wrong: my hint was only dedicated to show you a possible workaround until dc is able to handle the issue by itself.

clear: dc should work with any provider in general. but see my issue regarding hotmail.com. there is an unknown problem too :-( -- Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.

Am 15. Februar 2019 06:06:54 MEZ schrieb webratte notifications@github.com:

@csb0730 if you think the idea behind DC is a new toy for nerds your answer is fine.

But I believe the idea behind DC is a new secure AND user friendly messenger. In this case it's never a good idea to force users to search for hidden server settings which eventually exist for some mailproviders. A messenger have to work out of the box.

Users will not search for server settings outside of DC. They will find other messenger apps like Signal or WhatsApp where all can do inside the App.

BTW. my mailprovider (mailbox.org) offers not the option to limit storage time of messages. Should I change my mail provider to use DC?

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/deltachat/deltachat-core/issues/164#issuecomment-463908445

[testbird]

A fruitful discussion may involve to actually bringing up specific considerations or questions, and checking and seeing how they have been considered, or still need to be considered, for something like a reasonable current state of a roadmap or stepwise implementation plan. https://github.com/deltachat/deltachat-core/issues/120

[webratte]

@hpk42

I never said this will be easy.

The only thing I say is: DC is the only messenger I know where user gave to deal with server settings. The most user don't know how to create a sieve filter. Sounds strange but that's the true 😉

Users will be frustrated if they can't receive messages anymore because the mailbox storage is full.

As you know, with messengers people send tons of pictures and videos. Much more then with regular mail.

If you point this users to server settings they will left DC and go back to a messenger which works out of the box.

People are lazy.

Am 15. Februar 2019 10:09:20 MEZ schrieb holger krekel notifications@github.com:

On Thu, Feb 14, 2019 at 21:06 -0800, webratte wrote:

@csb0730 if you think the idea behind DC is a new toy for nerds your answer is fine.

This issue is about figuring out how to get rid of old messages stored in the server, right? If some people can modify server settings to achieve that like @csb0730 did, that's useful to point out. Some things are done much more easily at the provider site than with Delta Chat.

There are many different IMAP server configurations that Delta Chat needs to deal with and sometimes fails to deal with. Safely deleting messages from the server which are in DC's local store will take work and testing, as it's a destructive workflow (you might delete the wrong server side mails!). There are also people in at-risk situations who want to delete local chats, but keep the server state. There are UX questions involved on all three apps (android, desktop, ios), apart from API questions for deltachat-core.

Just mentioning this to clarify that what might look like an easy "Let's just introduce an option to do X" is usually embedded in a nexus of considerations, and many other developments and discussions happening. But i am optimistic we eventually implement a way to satisfy the needs expressed in this quite long issue thread :)

holger

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/deltachat/deltachat-core/issues/164#issuecomment-463962397

[testbird]

With the integrated ideas from deltachat/deltachat-core#120 and deltachat/deltachat-core#164 the basic auto-delete would always automatically ensure storage restrictions (keep free space) without surprising adverse effects or the user ever having to do anything.

The default could further be "100" for the "auto-archive of old messages" that would make current chats longer than 100 messages (these would stay archived locally, but get deleted from server).

[r10s]

i agree that some options are useful esp. for low-memory providers as in cuba.

DC is the only messenger I know where user gave to deal with server settings. The most user don't know how to create a sieve filter. Sounds strange but that's the true

Users will be frustrated if they can't receive messages anymore because the mailbox storage is full.

As you know, with messengers people send tons of pictures and videos. Much more then with regular mail. If you point this users to server settings they will left DC and go back to a messenger which works out of the box.

well, we should keep in mind that there are many "users" out there where storage space is not an issue and they will never have to deal with these settings.

eg. the initial example GMX allows 65 GB, Gmail uses the same dimensions afaik.

[testbird]

Better than options is a good default "storage restrictions" behaviour (#120).

[violoncelloCH]

If I'm right, it should be possible to query the IMAP quota. If we could check this we could propose to delete old messages on the server, if only a small amount of storage is remaining...

[webratte]

If it's right and you can query the IMAP quota so you could warn the user if storage will run out soon.

In this case a simple and secure way could a info like:

"Your mailbox is nearly full. You should delete old no more needed messages in inbox>deltatchat in your mailbox".

Of course a automated solution would be much more user friendly.

But a warning could help to be not frustrated.

[r10s]

If I'm right, it should be possible to query the IMAP quota.

not sure if this is widely supported, i would not rely on this. tested some random servers i am using for testing, most did not use QUOTA. possibly not even worth the effort.

this is the problem with so many IMAP extensions, btw. theoretically, everything is available - but in practice it is not :)

[webratte]

well, we should keep in mind that there are many "users" out there where storage space is not an issue and there will never have to deal with these settings. eg. the initial example GMX allows 65 GB, Gmail uses the same dimensions afaik.

OK, I understand, it's not easy this to implement. Still it's no way to say the GMX has 65 GB storage.

I have "only 2GB". Are you think I should switch to Gmail ;-P

I will attach 2 screenshots to illustrate what I mean.

On is my Quota for the mailbox. And one is my used storage in my DC folder after a few weeks and 4 !! Chatbuddys.

Don't forget, It's my mailbox. Not a standalone Chat account.

My son get everyday minimum 2 "funny" videoclips an of course lots of pictures of (I think more then 20-30 Chat partners) He is using WhatsApp. And he never missed a message because the Server storage is full.

And I think he is not a "extreme user". I thing it's a normal user. Similar behavior I can see by (adult) colleagues.

[adbenitez]

also my email provider don't provide this options at all, they don't even provide filters!! People here relies in the chatting apps (deltachat-like apps) to keep clean they INBOX

[adbenitez]

@r10s you can know if the QUOTA extension is supported since it is advertised on the server capabilities and then use it if available. I use "GETQUOTAROOT INBOX" to know my server quota, the response is in this format (STORAGE [S] 102400 MESSAGE [M] 15000) which means that there are not only a limit on storage space(100MB) but also in the number of messages(15000). Having such statistics available in the user is profile, in a nice way like a progress bar, would really help, I think if the QUOTA capability is not present then there are no need for it, it means that the server isn't interested in restricting the users space, so when it makes sense to worry about storage space the server will provide the QUOTA extension, otherwise how users would know their quota???

[r10s]

yip, we know of the IMAP CAPABILITIES flags, we use them eg. for checking if IMAP IDLE is supported.

i just wanted to point out that many IMAP extensions seems to be poorly spreaded :)

[adbenitez]

well if your server has no quota restrictions why would you need the QUOTA extension?? but if your server has quota restrictions it uses the QUOTA extension right??? or do you know a server with quota but without this capability??? how they let the users/clients to know their QUOTA?? I am just curious about this.

[r10s]

well, i fear there are many servers out there that have quota but do not have the QUOTA extension installed.

eg. my private testing server from a typical german shared hoster has quota but no QUOTA extension. well, of course, this might be the only one :)

[adbenitez]

if a server with a quota do not provide the QUOTA extension to users to check their quota, well that is a server misconfiguration of the server, Delta Chat just have to provide the functionality if available as the rest of email clients do and as DC is already doing for other capabilities.

[webratte]

Just to visualize why it's important to delete older messages a update of my current used server space only the Delta Chat folder (15 days after my last server space post):

It's growing from 5,1 MB to 33,6 MB. I'll keep you up to date...

There is no Video sended. Only the "normal" things like text messeges, fun pics and a few PDF documents. Power chatter will also send funny cat videos. I thing the "standard WhatsApp user" will reach 1GB in 2 or 3 month (or faster ;-) ).

So a mailbox will be full very fast.

[adbenitez]

in my case I get like 30MB per month and my server allow only up to 100MB this functionality is highly needed for users on the same server, since there are not easy/cheap way to cleanup the server.

[testbird]

From the discussion that gauged all the options to absolutely minimize server space usage (rather than keeping some limit), clients may only opt to "delete messages from server after they are read" (read-receipts still need to stay until all clients synced the state) https://support.delta.chat/t/workaround-make-new-delta-chat-installation-import-old-messages-from-imap-directory/262/14

[hpk42]

Closing this in favor of the followup #1206