search

internetstandards / Internet.nl

Internet standards compliance test suite
https://internet.nl
179 stars 38 forks source link

Better support slow tls_mail_smtp_starttls #1425

Open janwillemstegink opened 6 months ago

janwillemstegink commented 6 months ago

Email test: kconline.kcdekempen.nl (@bwbroersma already informed)

If the test of secure mail server connections does not end successfully:

  1. The DNS settings for email may still be evaluated and/or displayed;
  2. I don't get much information to get the hosting company in question to resolve.

image

image

bwbroersma commented 5 months ago

Ideally it should indeed provide more information about the error, e.g. timeout. The website test doesn't seem to have this kind of error behavior.

I just tested on 2 versions, v1.8.5.dev68-gd8e4822:

  1. 1218

    image

  2. Latest main (develop run locally) image

    However waiting a while (e.g. few minutes) and going to /mail/kconline.kcdekempen.nl/results will create a report: image

    I did the same on internet.nl, see the public result report (75%). The difference with the above in scoring is that in main RPKI is included in the scoring. It seems if the TLS test takes too long, it will error on the front-end, but still continue in the back-end, see the the probe below which takes 7 minutes and 3 seconds (23:04 - 16:01). So the problem is the test takes too long, and it is supported, but not on the front-end.

  3. probe main (develop run locally)

    $ make exec app "cmd=./manage.py probe --probe=tls_mail_smtp_starttls --domain=kconline.kcdekempen.nl" env=develop
    docker compose  --project-name=internetnl-develop exec --user root app ./manage.py probe --probe=tls_mail_smtp_starttls --domain=kconline.kcdekempen.nl
Batch enabled.
2024-06-02 11:16:01 DEBUG    - Running interface startup checks.
2024-06-02 11:16:01 DEBUG    - Loading autoconf into redis cache.
2024-06-02 11:16:01 DEBUG    - Performing batch startup checks.
ENABLE_BATCH is set for this server but the database is lacking the required indexes. Consider running `manage.py api_create_db_indexes`.
2024-06-02 11:16:02,358 INFO     - probe.py            :90   -            run_probe() - Performing tls_mail_smtp_starttls on kconline.kcdekempen.nl.
2024-06-02 11:16:02,358 DEBUG    - probe.py            :99   -            run_probe() - First retrieving mailservers
2024-06-02 11:16:02 DEBUG    - Attempting resolving of qname: kconline.kcdekempen.nl
2024-06-02 11:16:03 DEBUG    - Got data: {'done': True, 'secure': 0, 'bogus': 0, 'nxdomain': 0, 'data': <unbound.ub_data object at 0x7f0a577ea9a0>, 'rcode': 0}, retval: 0.
2024-06-02 11:16:03 DEBUG    - Attempting resolving of qname: _25._tcp.kconline.kcdekempen.nl.
2024-06-02 11:16:03 DEBUG    - Got data: {'done': True, 'secure': 0, 'bogus': 0, 'nxdomain': 1}, retval: 0.
2024-06-02 11:16:03,209 DEBUG    - probe.py            :101  -            run_probe() - Mailservers retrieved: [('kconline.kcdekempen.nl.', {}, <MxStatus.has_mx: 0>)]
2024-06-02 11:16:08 DEBUG    - SSL connect with ModernConnection to host 'kconline.kcdekempen.nl' at IP:port None:25 using SSL version TLSV1_3 invoked by __init__ > __init__ > __init__
2024-06-02 11:16:28 DEBUG    - SSL connect with DebugConnection to host 'kconline.kcdekempen.nl' at IP:port 157.90.129.50:25 using SSL version SSLV23 invoked by __init__ > from_conn > debug_conn
2024-06-02 11:16:49 DEBUG    - SSL connect with DebugConnection to host 'kconline.kcdekempen.nl' at IP:port 157.90.129.50:25 using SSL version SSLV23 invoked by __init__ > from_conn > check_client_reneg
2024-06-02 11:17:09 DEBUG    - SSL connect with DebugConnection to host 'kconline.kcdekempen.nl' at IP:port 157.90.129.50:25 using SSL version SSLV23 invoked by __init__ > from_conn > _check_ciphers
2024-06-02 11:17:29 DEBUG    - SSL connect with DebugConnection to host 'kconline.kcdekempen.nl' at IP:port 157.90.129.50:25 using SSL version SSLV2 invoked by __init__ > from_conn > _check_ciphers
2024-06-02 11:18:14 DEBUG    - SSL connect with ModernConnection to host 'kconline.kcdekempen.nl' at IP:port 157.90.129.50:25 using SSL version SSLV23 invoked by __init__ > from_conn > _check_ciphers
2024-06-02 11:18:34 DEBUG    - SSL connect with DebugConnection to host 'kconline.kcdekempen.nl' at IP:port 157.90.129.50:25 using SSL version TLSV1_1 invoked by __init__ > from_conn > check_protocol_versions
2024-06-02 11:18:55 DEBUG    - SSL connect with DebugConnection to host 'kconline.kcdekempen.nl' at IP:port 157.90.129.50:25 using SSL version TLSV1 invoked by __init__ > from_conn > check_protocol_versions
2024-06-02 11:19:15 DEBUG    - SSL connect with DebugConnection to host 'kconline.kcdekempen.nl' at IP:port 157.90.129.50:25 using SSL version SSLV3 invoked by __init__ > from_conn > check_protocol_versions
2024-06-02 11:19:35 DEBUG    - SSL connect with DebugConnection to host 'kconline.kcdekempen.nl' at IP:port 157.90.129.50:25 using SSL version SSLV2 invoked by __init__ > from_conn > check_protocol_versions
2024-06-02 11:20:20 DEBUG    - SSL connect with DebugConnection to host 'kconline.kcdekempen.nl' at IP:port 157.90.129.50:25 using SSL version SSLV23 invoked by __init__ > from_conn > check_dh_params
2024-06-02 11:20:40 DEBUG    - SSL connect with DebugConnection to host 'kconline.kcdekempen.nl' at IP:port 157.90.129.50:25 using SSL version SSLV23 invoked by __init__ > from_conn > check_dh_params
2024-06-02 11:21:00 DEBUG    - SSL connect with ModernConnection to host 'kconline.kcdekempen.nl' at IP:port 157.90.129.50:25 using SSL version TLSV1_3 invoked by __init__ > from_conn > sha2_supported_or_na
2024-06-02 11:21:21 DEBUG    - SSL connect with ModernConnection to host 'kconline.kcdekempen.nl' at IP:port 157.90.129.50:25 using SSL version TLSV1_2 invoked by __init__ > from_conn > sha2_supported_or_na
2024-06-02 11:21:41 DEBUG    - SSL connect with DebugConnection to host 'kconline.kcdekempen.nl' at IP:port 157.90.129.50:25 using SSL version SSLV23 invoked by __init__ > from_conn > _check_ciphers
2024-06-02 11:22:01 DEBUG    - Current cipher_order == CipherOrderStatus.good, will only test when this is: CipherOrderStatus.good.
2024-06-02 11:22:01 DEBUG    - Testing cipher order for TLS1.2
2024-06-02 11:22:01 DEBUG    - SSL connect with DebugConnection to host 'kconline.kcdekempen.nl' at IP:port 157.90.129.50:25 using SSL version TLSV1_2 invoked by __init__ > from_conn > check_cipher_order
2024-06-02 11:22:21 DEBUG    - Retrieved ciphers: ['DHE-RSA-AES256-GCM-SHA384', 'ECDHE-RSA-AES256-GCM-SHA384', 'DHE-RSA-AES128-GCM-SHA256'].
2024-06-02 11:22:21 DEBUG    - SSL connect with DebugConnection to host 'kconline.kcdekempen.nl' at IP:port 157.90.129.50:25 using SSL version SSLV23 invoked by __init__ > from_conn > dup
2024-06-02 11:22:42 DEBUG    - SSL connect with DebugConnection to host 'kconline.kcdekempen.nl' at IP:port 157.90.129.50:25 using SSL version SSLV23 invoked by __init__ > from_conn > _check_ciphers
2024-06-02 11:23:02 DEBUG    - Returning. order tested: CipherOrderStatus.good, order score: 10
2024-06-02 11:23:02 DEBUG    - Attempting resolving of qname: _25._tcp.kconline.kcdekempen.nl.
2024-06-02 11:23:03 DEBUG    - Got data: {'done': True, 'secure': 0, 'bogus': 0, 'nxdomain': 1}, retval: 0.
2024-06-02 11:23:04,019 INFO     - probe.py            :113  -            run_probe() - Retrieved return value: ('smtp_starttls', {'kconline.kcdekempen.nl.': {'tls_enabled': True, 'tls_enabled_score': 10, 'prots_bad': [], 'prots_phase_out': [], 'prots_good': ['TLS 1.3'], 'prots_sufficient': ['TLS 1.2'], 'prots_score': 10, 'ciphers_bad': [], 'ciphers_phase_out': [], 'ciphers_score': 10, 'cipher_order_score': 10, 'cipher_order': <CipherOrderStatus.good: 1>, 'cipher_order_violation': [], 'secure_reneg': True, 'secure_reneg_score': 10, 'client_reneg': False, 'client_reneg_score': 10, 'compression': False, 'compression_score': 10, 'dh_param': '2048', 'ecdh_param': '521', 'fs_bad': ['DH-2048'], 'fs_phase_out': [], 'fs_score': 0, 'zero_rtt_score': 10, 'zero_rtt': <ZeroRttStatus.good: 1>, 'kex_hash_func': <KexHashFuncStatus.good: 1>, 'kex_hash_func_score': 10, 'tls_cert': True, 'chain': ['nvmefalk02.040services.net', 'R3'], 'trusted': 0, 'trusted_score': 10, 'pubkey_bad': [], 'pubkey_phase_out': [], 'pubkey_score': 10, 'sigalg_bad': {}, 'sigalg_score': 10, 'hostmatch_bad': ['nvmefalk02.040services.net'], 'hostmatch_score': 10, 'dane_score': 0, 'dane_status': <DaneStatus.none: 2>, 'dane_log': '', 'dane_records': [], 'dane_rollover': False}})
2024-06-02 11:23:04,022 INFO     - probe.py            :114  -            run_probe() - Done

So at least internet.nl should show 1, or have some 'extended' wait, to show 2. For now this can manually be hacked, but it seems the new sslyze PR handles the time-out better than the current code.