search

deltachat / deltachat-core-rust

Delta Chat Rust Core library, used by Android/iOS/desktop apps, bindings and bots 📧
https://delta.chat/en/contribute
Other
680 stars 87 forks source link

Same IMAP and SMTP parameters configured manually in old and newly created profile lead to different connection attempts #5996

Open gerryfrancis opened 2 months ago

gerryfrancis commented 2 months ago
r10s commented 1 month ago

Manually configured IMAP and SMTP parameters seem to be overruled/reconfigured by other ones.

but was it working at the end?

delta chat trying different servers and configurations is a new feature - might be that also other than the "entered" ones are tried. if so i would regard that as a feature as well - if the entered server is not reachable or slower in some network, and there is another option, it seems good if that is used.

reconfiguration is even done "on the fly" btw - so if you enter a wifi that blocks one port but allows another one, things continue working

gerryfrancis commented 1 month ago

but was it working at the end?

@r10s Yes, it worked, but the same IMAP and SMTP ports that are specified for TLS by Posteo were also used for STARTTLS, it seems:

used_account_settings=anonymous@posteo.at imap:[posteo.de:993:tls:anonymous@posteo.de, posteo.de:993:starttls:anonymous@posteo.de] smtp:[posteo.de:465:tls:anonymous@posteo.de, posteo.de:465:starttls:anonymous@posteo.de] provider:none cert_automatic

And that is not what is registered for Posteo in the provider database, they have different ports for STARTTLS: https://github.com/deltachat/provider-db/blob/master/_providers/posteo.md

So, automatic reconfiguration might have found a setup that works, but is not intended to be used in the field.

gerryfrancis commented 1 month ago

To add: Posteo recommend to use TLS instead of STARTTLS: https://posteo.de/en/help/how-do-i-set-up-posteo-in-an-email-client-pop3-imap-and-smtp

We recommend using TLS instead of STARTTLS if possible.

Instead, automatic reconfiguration sets STARTTLS to use TLS IMAP and SMTP ports (see above), and that seems to be pretty wrong.

hpk42 commented 1 month ago

On Thu, Oct 03, 2024 at 02:39 -0700, gerryfrancis wrote:

To add: Posteo recommend to use TLS instead of STARTTLS: https://posteo.de/en/help/how-do-i-set-up-posteo-in-an-email-client-pop3-imap-and-smtp

We recommend using TLS instead of STARTTLS if possible.

Instead, automatic reconfiguration sets STARTTLS for TLS IMAP and SMTP ports (see above), and that seems to be pretty wrong.

i think these days DC would "automagically" reconfigure if posteo decides to forbid STARTLS some day.

Not sure if "TLS" could generally be prefererred over starttls. Auto-reconfiguration is a relative recent change in core.

iequidoo commented 1 month ago

Instead, automatic reconfiguration sets STARTTLS to use TLS IMAP and SMTP ports (see above), and that seems to be pretty wrong.

I checked with openssl s_client -showcerts -connect posteo.de:993 -starttls imap, it doesn't really work. The same for SMTP, STARTTLS only works if 143 and 587 ports are used. Apparently, these STARTTLS configurations were added because you specified IMAP/SMTP Security = Automatic. Everything works because TLS configurations are used (they are in the heads of the lists from your log), not STARTTLS. But i agree that it's at least strange that they are present in used_account_settings.

EDIT:

This is a log for your old account as far as i can tell, the ports are correct there for STARTTLS. Would be more interesting to see the log for the newly created account.

gerryfrancis commented 1 month ago

This is a log for your old account as far as i can tell, the ports are correct there for STARTTLS. Would be more interesting to see the log for the newly created account.

@iequidoo This is the log of a newly created account: deltachat-log-20241015-133411_2.txt

iequidoo commented 1 month ago

Still, it's not clear how incorrect STARTTLS ports got into configuration, in the beginning of the log i just see this: used_account_settings=anonymous@posteo.at imap:[posteo.de:993:tls:anonymous@posteo.de, posteo.de:993:starttls:anonymous@posteo.de] smtp:[posteo.de:465:tls:anonymous@posteo.de, posteo.de:465:starttls:anonymous@posteo.de] provider:none cert_automatic I guess they haven't been even tried by the core, but it just adds them because you specified Automatic security. Need to look into the code probably.

iequidoo commented 3 weeks ago

The logic that added the above ports to the configuration is here: https://github.com/deltachat/deltachat-core-rust/blob/92c6dd483cc566b5de3cfc5bca25143520a16aaa/src/configure/server_params.rs#L108-L120

So, @gerryfrancis, this is because you set the ports manually as far as i see on your screenshot. Posteo uses the standard ports for TLS and STARTTLS, so you don't need to specify them manually. Please try to leave them empty and share a new log. I guess Delta Chat will use the correct ports then.

gerryfrancis commented 3 weeks ago

@iequidoo Thank you for looking into the code, I can confirm that the correct ports have been determined after I reconfigured the profile without entering the port numbers for IMAP and SMTP:

entered_account_settings=anonymous@posteo.at imap:anonymous@posteo.de:***:posteo.de:0:Automatic:AUTH_NORMAL smtp:anonymous@posteo.de:***:posteo.de:0:Automatic:AUTH_NORMAL cert_automatic used_account_settings=anonymous@posteo.at imap:[posteo.de:993:tls:anonymous@posteo.de, posteo.de:143:starttls:anonymous@posteo.de] smtp:[posteo.de:465:tls:anonymous@posteo.de, posteo.de:587:starttls:anonymous@posteo.de] provider:none cert_automatic

However, I think the possibility of unknowingly misconfiguring the same ports for TLS and STARTTLS is a bug and must be avoided. Maybe another connection check in the background can be introduced to verify and adjust the configuration if necessary.

iequidoo commented 3 weeks ago

However, I think the possibility of unknowingly misconfiguring the same ports for TLS and STARTTLS is a bug and must be avoided. Maybe another connection check in the background can be introduced to verify and adjust the configuration if necessary.

The ports for STARTTLS were added to the configuration because Delta Chat didn't try them when configuring because the TLS configuration just worked. If TLS stops working for some reason, Delta Chat will try STARTTLS, so we don't need to check it in advance. Even if we check the STARTTLS configuration in advance and it doesn't work, what should we do, try the standard STARTTLS ports instead? But the user specified the exact ports, so such logic looks unclear. Another option is to warn the user that STARTTLS doesn't work (EDIT: it will never work if TLS works, because it's another protocol), but i think most of the users aren't even interested in which configuration they use and the remaining are just ok with not having a working STARTTLS configuration if TLS works.

gerryfrancis commented 3 weeks ago

If TLS stops working for some reason, Delta Chat will try STARTTLS, so we don't need to check it in advance.

The issue is about the same ports configured for TLS and STARTTLS by accident when just the TLS ones are specified manually. The app will not be able to connect using STARTTLS either when TLS suddenly does not work anymore.

Even if we check the STARTTLS configuration in advance and it doesn't work, what should we do, try the standard STARTTLS ports instead?

This, or just do not configure STARTTLS at all to avoid applying the wrong ports to it. If TLS connectivity died in case of failure, a falsely configured STARTTLS fallback would be even worse. In my opinion, the standard ports for STARTTLS should be examined and used in cases of TLS malfunction, but it is just an idea, of course.

iequidoo commented 3 weeks ago

The issue is about the same ports configured for TLS and STARTTLS by accident when just the TLS ones are specified manually.

Delta Chat could detect that the specified ports are the standard TLS/STRATTLS ones and use the standard ports for all protocols in this case (as if the user left the fields empty), but i'd say such a logic is unclear. I'd expect that the program only connects to the specified ports. I think this issue is rather an UI one, the user actually wants to use the standard ports for everything, but for some reason (e.g. their provider recommends that) specifies ports only for TLS or STARTTLS. Some hints should be added, like "Leave empty to use 993/TLS and 143/STARTTLS".

And i don't think we really need 4 fields to specify ports for all protocols.

EDIT:

or just do not configure STARTTLS at all to avoid applying the wrong ports to it.

The problem is that we can't know if the specified ports are wrong or were specified on purpose.

gerryfrancis commented 3 weeks ago

In my eyes, the logic should be like this:

Instead, we accidentally determine a configuration which uses the same ports for both, TLS and STARTTLS:

used_account_settings=anonymous@posteo.at imap:[posteo.de:993:tls:anonymous@posteo.de, posteo.de:993:starttls:anonymous@posteo.de] smtp:[posteo.de:465:tls:anonymous@posteo.de, posteo.de:465:starttls:anonymous@posteo.de] provider:none cert_automatic

Delta Chat could detect that the specified ports are the standard TLS/STRATTLS ones and use the standard ports for all protocols in this case (as if the user left the fields empty), but i'd say such a logic is unclear. I'd expect that the program only connects to the specified ports.

So do I, but in case of lost connectivity I think it would be legal to determine another way to connect to the server, just like we do with cached DNS results.

And i don't think we really need 4 fields to specify ports for all protocols.

Copy that. However, using the same ports for all secure transmission protocols, like we apparently apply (see used_account_settings above) just because the user could not specify other ones, cannot be the "right" solution either.

So, according to this example, I suggest to shorten the configuration as follows:

used_account_settings=anonymous@posteo.at imap:[posteo.de:993:tls:anonymous@posteo.de] smtp:[posteo.de:465:tls:anonymous@posteo.de] provider:none cert_automatic

This configuration could be adjusted to add standard STARTTLS ports later in case the specified TLS ports do not work anymore one day:

used_account_settings=anonymous@posteo.at imap:[posteo.de:993:tls:anonymous@posteo.de, posteo.de:143:starttls:anonymous@posteo.de] smtp:[posteo.de:465:tls:anonymous@posteo.de, posteo.de:587:starttls:anonymous@posteo.de] provider:none cert_automatic

I would not say it was "right" to do that from the view of a programmer, but it could help to maintain app connectivity, which probably might be the goal for the most common users.

iequidoo commented 3 weeks ago

However, using the same ports for all secure transmission protocols, like we apparently apply (see used_account_settings above) just because the user could not specify other ones, cannot be the "right" solution either.

Using the same ports for STARTTLS may be helpful if later the server decides to switch from TLS to STARTTLS (or vice versa which is more likely). But this is probably for the case of nonstandard ports. In case of the standard ports the server probably will change the ports also, but then the user just shouldn't specify them and that should be clear in the UI. So, the first option is to improve the UI. Another option is, well, to connect to the standard STARTTLS ports, but for me connecting to any ports other than specified by the user is strange and it breaks the scenario described above. To keep that scenario working we can add two STARTTLS configurations (with the user-specified ports and the standard ports), but i'd prefer to just improve the UI.