search

emqx / emqx-rel

Release Project for EMQX Broker prior to 4.3. Newer releases are built here: https://github.com/emqx/emqx
https://www.emqx.com
Apache License 2.0
190 stars 221 forks source link

Can't use ssl to connect to emqx docker #527

Closed chunzhenzyd closed 4 years ago

chunzhenzyd commented 4 years ago

Environment

Description

I followed https://blog.csdn.net/emqx_broker/article/details/99439312 to create self signed certs in Centos7 server 192.168.17.53.

root@192.168.17.53 /opt/ssl

pwd /opt/ssl root@192.168.17.53 /opt/ssl ls -lrt certs/ 总用量 56 -rw-r--r--. 1 root root 1675 6月 16 11:23 root-ca.key -rw-r--r--. 1 root root 1212 6月 16 11:23 root-cacert.pem -rw-r--r--. 1 root root 1675 6月 16 11:25 client.key -rw-r--r--. 1 root root 948 6月 16 11:25 client-cert.csr -rw-r--r--. 1 root root 3967 6月 16 11:33 client-cert.pem -rw-r--r--. 1 root root 1675 6月 16 11:34 server.key -rw-r--r--. 1 root root 948 6月 16 11:34 server-cert.csr -rw-r--r--. 1 root root 3967 6月 16 11:34 server-cert.pem -rw-r--r--. 1 root root 1679 6月 16 15:05 second-ca.key -rw-r--r--. 1 root root 952 6月 16 15:05 second-cacert.csr -rw-r--r--. 1 root root 4194 6月 16 15:06 second-cacert.pem -rw-r--r--. 1 root root 5406 6月 16 15:07 cacert.pem

Then I run emqx container with: docker run -d -u root --restart=always --name mqtt --network ims -p 1883:1883 -p 8083:8083 -p 8883:8883 -p 18083:18083 \ --log-driver=json-file --log-opt max-size=20m --log-opt max-file=5 \ -e EMQX_NAME=mqtt -e EMQX_ZONE__EXTERNAL__MAX_MQUEUE_LEN=0 -e EMQX_ZONE__INTERNAL__MAX_MQUEUE_LEN=0 \ -e EMQX_MQTT__RETAIN_AVAILABLE=false -e EMQX_ALLOW_ANONYMOUS=false -e EMQX_ZONE__INTERNAL__ALLOW_ANONYMOUS=false \ -e EMQX_LISTENER__SSL__EXTERNAL__KEYFILE=etc/certs/server.key \ -e EMQX_LISTENER__SSL__EXTERNAL__CERTFILE=etc/certs/server-cert.pem \ -e EMQX_LISTENER__SSL__EXTERNAL__CACERTFILE=etc/certs/cacert.pem \ -e EMQX_LOG__CONSOLE__LEVEL=debug \ -v /home/clear/ims/conf/mqtt/acl.conf:/opt/emqx/etc/acl.conf \ -v /opt/ssl/certs:/opt/emqx/etc/certs \ -v /etc/localtime:/etc/localtime:ro \ emqx/emqx:latest

Then I try to connect from my local computer(192.168.17.253) with MQTTX(v1.3.1):

  1. copy certs/cacert.pem from 192.168.17.53 to 192.168.17.253:~/Downloads/
  2. set Host : mqtts://192.168.17.53, Port : 8883
  3. set SSL/TLS : true, Certificate : 'Self signed', Certificates.CA Files : ~/Downloads/certs/cacert.pem

But it failed when connecting: (mqtt@172.18.0.3)1> 15:55:15.459 [debug] MQTT(192.168.17.253:61269): RECV <<16>> (mqtt@172.18.0.3)1> 15:55:30.461 [debug] MQTT(192.168.17.253:61269): Terminated for {shutdown,idle_timeout}

chunzhenzyd commented 4 years ago

When turn on 'Strict validate Certificate', it will pop an error: [ERR_TLS_CERT_ALTNAME_INVALID]: Hostname/IP does not match certificate's altnames: IP: 192.168.17.53 is not in the cert's list:

Rory-Z commented 4 years ago

Hi, @chunzhenzyd It seems that your certificate file can only be used on 192.168.17.53, and each docker container has a separate IP. The IP of docker container is not 192.168.17.53, so the error occurs. I suggest you use the host network mode to deploy the docker container, like this:

docker run --networl host ......
chunzhenzyd commented 4 years ago

Hi, @chunzhenzyd It seems that your certificate file can only be used on 192.168.17.53, and each docker container has a separate IP. The IP of docker container is not 192.168.17.53, so the error occurs. I suggest you use the host network mode to deploy the docker container, like this:

docker run --networl host ......

Hi @zhanghongtong,

I use cfssl re-create certificate files, as you mentioned that "each docker container has a separate IP", the IP of docker container is "172.18.0.3":

cfssl print-defaults config > config.json
cfssl print-defaults csr > csr.json

cat > ca-config.json <<EOF
{
    "signing":{
        "default":{
            "expiry":"87600h"
        },
        "profiles":{
            "kubernetes":{
                "expiry":"87600h",
                "usages":[
                    "signing",
                    "key encipherment",
                    "server auth",
                    "client auth"
                ]
            }
        }
    }
}
EOF

cat > ca-csr.json <<EOF
{
    "CN":"kubernetes",
    "key":{
        "algo":"rsa",
        "size":2048
    },
    "names":[
        {
            "C":"CN",
            "L":"Hebei",
            "ST":"Zhangjiakou",
            "O":"k8s",
            "OU":"System"
        }
    ]
}
EOF

cfssl gencert -initca ca-csr.json | cfssljson -bare ca -

cat > server-csr.json << EOF
{
    "CN":"kubernetes",
    "hosts":[
        "127.0.0.1",
        "192.168.17.53",
        "172.18.0.3",
        "mqtt",
        "*.cleartv.cn"
    ],
    "key":{
        "algo":"rsa",
        "size":2048
    },
    "names":[
        {
            "C":"CN",
            "L":"Hebei",
            "ST":"Zhangjiakou",
            "O":"k8s",
            "OU":"System"
        }
    ]
}
EOF

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson -bare server

And restart emqx docker, then connect from mqttx ( 192.168.17.253, using /certs/ca.pem ) , no mqttx client error poped.

root@192.168.17.53 /opt/ssl
> docker logs -f mqtt
node.max_ports=1048576
listener.tcp.external.acceptors=64
listener.ssl.external.certfile=etc/certs/server.pem
allow_anonymous=false
listener.ssl.external.acceptors=32
listener.ssl.external.keyfile=etc/certs/server-key.pem
node.process_limit=2097152
node.max_ets_tables=2097152
mqtt.retain_available=false
listener.ws.external.acceptors=16
listener.ssl.external.cacertfile=etc/certs/ca.pem
zone.internal.max_mqueue_len=0
log.console.level=debug
node.name=mqtt@172.18.0.3
zone.internal.allow_anonymous=false
zone.external.max_mqueue_len=0
emqx 3.0 is started successfully!
    options: []
=INFO REPORT==== 16-Jun-2020::18:59:34.321777 ===
    msg: event=start
    options: []
=INFO REPORT==== 16-Jun-2020::18:59:34.324103 ===
    alarm_handler: {set,{system_memory_high_watermark,[]}}
18:59:34.336 [debug] Lager installed handler {lager_file_backend,"log/error.log"} into lager_event
18:59:34.336 [debug] Lager installed handler {lager_syslog_backend,{"emqx",local0}} into lager_event
18:59:34.336 [debug] Lager installed handler error_logger_lager_h into error_logger
Starting emqx on node mqtt@172.18.0.3
Load emqx_mod_presence module successfully.
Start http:management listener on 8080 successfully.
18:59:34.831 [debug] Lager installed handler lager_backend_throttle into lager_event
18:59:34.978 [info] Started Apps: [clique,minirest,emqx_management]
18:59:34.978 [info] Load plugin emqx_management successfully
18:59:34.978 [info] Started Apps: [recon,emqx_recon]
18:59:34.978 [info] Load plugin emqx_recon successfully
18:59:35.016 [info] Started Apps: [emqx_retainer]
18:59:35.016 [info] Load plugin emqx_retainer successfully
Start http:dashboard listener on 18083 successfully.
18:59:35.054 [info] Started Apps: [emqx_dashboard]
18:59:35.054 [info] Load plugin emqx_dashboard successfully
emqx_delayed_publish is loaded.
18:59:35.055 [info] Started Apps: [emqx_delayed_publish]
18:59:35.055 [info] Load plugin emqx_delayed_publish successfully
Start mqtt:tcp listener on 127.0.0.1:11883 successfully.
Start mqtt:tcp listener on 0.0.0.0:1883 successfully.
Start mqtt:ws listener on 0.0.0.0:8083 successfully.
Start mqtt:ssl listener on 0.0.0.0:8883 successfully.
Start mqtt:wss listener on 0.0.0.0:8084 successfully.
EMQ X Broker 3.0 is running now!
Eshell V10.0.6  (abort with ^G)
(mqtt@172.18.0.3)1> ['2020-06-16T10:59:35Z']:waiting emqx
['2020-06-16T10:59:41Z']:emqx start
19:05:22.379 [debug] MQTT(192.168.17.253:61877): RECV <<16>>
(mqtt@172.18.0.3)1> 19:05:37.380 [debug] MQTT(192.168.17.253:61877): Terminated for {shutdown,idle_timeout}
chunzhenzyd commented 4 years ago

Hi @zhanghongtong 

root@192.168.17.53 /opt/ssl
> ls -lrt /opt/ssl/certs/
总用量 44
-rw-r--r--. 1 root root 1363 6月  16 18:58 ca.pem
-rw-------. 1 root root 1675 6月  16 18:58 ca-key.pem
-rw-r--r--. 1 root root  256 6月  16 18:58 ca-csr.json
-rw-r--r--. 1 root root 1005 6月  16 18:58 ca.csr
-rw-r--r--. 1 root root  379 6月  16 18:58 ca-config.json
-rw-r--r--. 1 root root 1468 6月  16 18:58 server.pem
-rw-------. 1 root root 1679 6月  16 18:58 server-key.pem
-rw-r--r--. 1 root root  384 6月  16 18:58 server-csr.json
-rw-r--r--. 1 root root 1094 6月  16 18:58 server.csr
-rw-r--r--. 1 root root  287 6月  16 18:58 csr.json
-rw-r--r--. 1 root root  567 6月  16 18:58 config.json
root@192.168.17.53 /opt/ssl
> openssl x509 -in certs/server.pem -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            4b:77:cf:41:92:43:b5:0f:82:4c:ff:f4:17:25:2c:50:89:5c:7c:6a
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=CN, ST=Zhangjiakou, L=Hebei, O=k8s, OU=System, CN=kubernetes
        Validity
            Not Before: Jun 16 10:53:00 2020 GMT
            Not After : Jun 14 10:53:00 2030 GMT
        Subject: C=CN, ST=Zhangjiakou, L=Hebei, O=k8s, OU=System, CN=kubernetes
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:99:44:b7:16:6e:bc:3b:e0:5d:55:89:e1:97:6d:
                    16:77:f9:0a:81:5e:ba:7b:e0:e8:43:ba:6a:3b:ab:
                    e0:95:ad:05:35:9f:2d:ac:bf:75:4b:33:62:77:a9:
                    f9:74:d8:58:b7:8d:3e:ef:70:f5:98:da:b6:12:2b:
                    0b:9e:e9:a8:ad:ad:35:cb:1c:9a:95:b1:7f:e5:a4:
                    23:fb:7e:49:11:34:68:94:8f:49:0f:e1:fa:4f:5c:
                    cf:91:e7:a8:b4:ae:bd:42:02:fa:e2:2a:2f:e2:16:
                    39:3e:f2:34:82:5e:40:a8:5b:8f:60:48:f1:12:98:
                    3b:51:86:59:e0:26:52:e5:1c:95:b3:19:7f:f8:c5:
                    80:85:52:71:c4:88:42:04:7f:44:28:2d:17:8a:6a:
                    0a:fe:9a:d4:1c:52:30:87:f4:fc:03:a7:5e:dd:79:
                    cb:33:b8:82:ca:45:a3:5b:8e:1a:17:90:60:6a:65:
                    56:5c:a3:70:f7:6f:cd:68:19:28:41:e4:a9:49:8f:
                    95:f0:d1:2f:a6:3a:55:4e:7a:1a:4a:46:f9:39:eb:
                    53:7e:5d:b8:bc:38:75:dd:63:1c:6c:cb:7d:89:95:
                    c6:be:8f:7f:fe:71:48:c7:c3:83:de:c2:42:47:8d:
                    d2:6e:aa:56:66:99:c5:58:d2:98:bd:2b:cc:24:20:
                    1e:55
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage:
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Subject Key Identifier:
                04:DE:C4:81:B1:5F:6D:39:27:81:CB:10:B1:CF:75:EA:10:11:3C:1A
            X509v3 Authority Key Identifier:
                keyid:BD:54:7F:65:43:D2:8D:23:4C:0D:A5:66:45:FE:2B:99:6D:F4:1D:78

            X509v3 Subject Alternative Name:
                DNS:mqtt, DNS:*.cleartv.cn, IP Address:127.0.0.1, IP Address:192.168.17.53, IP Address:172.18.0.3
    Signature Algorithm: sha256WithRSAEncryption
         01:e3:c7:a8:d3:9c:76:17:af:66:dd:b5:94:9e:55:b5:1a:05:
         b4:6a:4b:b8:95:3b:90:f5:13:0d:93:d1:96:f6:5e:26:8b:7e:
         7b:f4:18:31:b5:82:c8:76:92:e8:97:0e:c4:2b:81:6f:2f:36:
         46:28:fe:a6:6e:35:5f:44:a6:13:08:17:11:56:bc:7d:2f:14:
         fc:14:da:cd:2a:84:16:68:5b:60:ba:b7:00:c5:c6:33:9c:c7:
         cb:7e:10:f0:76:b7:42:02:ce:f8:f9:60:65:0a:18:67:fc:c3:
         3b:d5:d7:0f:f9:75:f8:84:31:3c:00:e9:47:1c:21:66:c1:1d:
         34:f7:ad:24:7e:2f:df:49:1e:57:a7:07:ec:c1:f1:23:19:24:
         8a:76:d5:2e:d9:a4:29:b9:39:6d:00:0d:a8:ab:9a:1b:06:dd:
         64:d9:08:b7:6f:b9:8e:fc:62:e8:1b:67:36:1c:28:6e:a2:e9:
         bc:14:40:07:80:fd:51:5a:ef:45:09:b6:60:3e:f2:a7:8f:09:
         73:5e:36:8e:3a:59:d8:bb:75:ff:df:e7:01:6f:20:08:8a:ff:
         d3:20:aa:d2:ab:c7:56:51:82:ee:7d:db:f3:fe:32:74:7f:34:
         43:e9:f6:0a:29:11:c7:5d:d4:1f:13:30:de:9b:92:45:a6:17:
         ca:3b:14:3e
chunzhenzyd commented 4 years ago

Hi @zhanghongtong

(At 192.168.17.253 mqttx) I changed the ca.pem to another 'CA File', then an error poped:

Error: unable to verify the first certificate

So, I think ca.pem has no problem( No error poped ), but emqx docker container has a bug.

Rory-Z commented 4 years ago

Hi @zhanghongtong

(At 192.168.17.253 mqttx) I changed the ca.pem to another 'CA File', then an error poped:

Error: unable to verify the first certificate

So, I think ca.pem has no problem( No error poped ), but emqx docker container has a bug.

Hi, @chunzhenzyd This error appears to be caused by an MQTTX certificate error, I don't understand why you think the emqx docker container has a bug

chunzhenzyd commented 4 years ago

Hi @zhanghongtong

I know 'Error: unable to verify the first certificate' was caused by an MQTTX certificate error, becase I changed the 'CA File' to a wrong ca certificate file(errca.pem). ca.pem is the right ca certificate file, but still can't connect emqx, and no error poped. I do this just for checking whether is the ca.pem is correct.

Two certificate files: 1. ca.pem, 2. errca.pem use 1 (ca.pem), no error poped, emqx docker log: 19:05:22.379 [debug] MQTT(192.168.17.253:61877): RECV <<16>> (mqtt@172.18.0.3)1> 19:05:37.380 [debug] MQTT(192.168.17.253:61877): Terminated for {shutdown,idle_timeout} use 2 (errca.pem), 'Error: unable to verify the first certificate' poped.

chunzhenzyd commented 4 years ago

If you have the correct way to use self signed certificate to connect emqx docker, please share, and I will follow and try.

why I think the emqx docker container has a bug:

  1. I have tried to use self signed certificate to connect mosquitto( not docker container ), and it's ok.
  2. use the same way to connect emqx docker container, emqx docker error log: (mqtt@172.18.0.3)1> 19:05:22.379 [debug] MQTT(192.168.17.253:61877): RECV <<16>> (mqtt@172.18.0.3)1> 19:05:37.380 [debug] MQTT(192.168.17.253:61877): Terminated for {shutdown,idle_timeout}
chunzhenzyd commented 4 years ago

@emqplus @velimir @ngjaying Anyone help?

spring2maz commented 4 years ago

You can validate broker certificate by:

openssl s_client -showcerts -CAfile /path/to/cacert.pem -connect 192.168.17.53:8883

Verify return code should be 0 if everything goes fine. Or even set --insecure in mosquitto client options, so client do not have to provide certificates, also to skip server host validation (this is to check if the issue is indeed in TLS handshake).

The 15 seconds delay of idle_timeout debug looks to me is something happened after TLS connection has been successfully established.

chunzhenzyd commented 4 years ago

@spring2maz

Hi,

Thanks for your suggestion.

Output of certificate validation:

> openssl s_client -showcerts -CAfile /Users/zyd/Downloads/certs/ca.pem -connect 192.168.17.53:8883
CONNECTED(00000003)
depth=1 C = CN, ST = Zhangjiakou, L = Hebei, O = k8s, OU = System, CN = kubernetes
verify return:1
depth=0 C = CN, ST = Zhangjiakou, L = Hebei, O = k8s, OU = System, CN = kubernetes
verify return:1
---
Certificate chain
 0 s:/C=CN/ST=Zhangjiakou/L=Hebei/O=k8s/OU=System/CN=kubernetes
   i:/C=CN/ST=Zhangjiakou/L=Hebei/O=k8s/OU=System/CN=kubernetes
-----BEGIN CERTIFICATE-----
MIIEDjCCAvagAwIBAgIUS3fPQZJDtQ+CTP/0FyUsUIlcfGowDQYJKoZIhvcNAQEL
BQAwZzELMAkGA1UEBhMCQ04xFDASBgNVBAgTC1poYW5namlha291MQ4wDAYDVQQH
EwVIZWJlaTEMMAoGA1UEChMDazhzMQ8wDQYDVQQLEwZTeXN0ZW0xEzARBgNVBAMT
Cmt1YmVybmV0ZXMwHhcNMjAwNjE2MTA1MzAwWhcNMzAwNjE0MTA1MzAwWjBnMQsw
CQYDVQQGEwJDTjEUMBIGA1UECBMLWmhhbmdqaWFrb3UxDjAMBgNVBAcTBUhlYmVp
MQwwCgYDVQQKEwNrOHMxDzANBgNVBAsTBlN5c3RlbTETMBEGA1UEAxMKa3ViZXJu
ZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAJlEtxZuvDvgXVWJ
4ZdtFnf5CoFeunvg6EO6ajur4JWtBTWfLay/dUszYnep+XTYWLeNPu9w9ZjathIr
C57pqK2tNcscmpWxf+WkI/t+SRE0aJSPSQ/h+k9cz5HnqLSuvUIC+uIqL+IWOT7y
NIJeQKhbj2BI8RKYO1GGWeAmUuUclbMZf/jFgIVSccSIQgR/RCgtF4pqCv6a1BxS
MIf0/AOnXt15yzO4gspFo1uOGheQYGplVlyjcPdvzWgZKEHkqUmPlfDRL6Y6VU56
GkpG+TnrU35duLw4dd1jHGzLfYmVxr6Pf/5xSMfDg97CQkeN0m6qVmaZxVjSmL0r
zCQgHlUCAwEAAaOBsTCBrjAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYB
BQUHAwEGCCsGAQUFBwMCMAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFATexIGxX205
J4HLELHPdeoQETwaMB8GA1UdIwQYMBaAFL1Uf2VD0o0jTA2lZkX+K5lt9B14MC8G
A1UdEQQoMCaCBG1xdHSCDCouY2xlYXJ0di5jbocEfwAAAYcEwKgRNYcErBIAAzAN
BgkqhkiG9w0BAQsFAAOCAQEAAePHqNOcdhevZt21lJ5VtRoFtGpLuJU7kPUTDZPR
lvZeJot+e/QYMbWCyHaS6JcOxCuBby82Rij+pm41X0SmEwgXEVa8fS8U/BTazSqE
FmhbYLq3AMXGM5zHy34Q8Ha3QgLO+PlgZQoYZ/zDO9XXD/l1+IQxPADpRxwhZsEd
NPetJH4v30keV6cH7MHxIxkkinbVLtmkKbk5bQANqKuaGwbdZNkIt2+5jvxi6Btn
NhwobqLpvBRAB4D9UVrvRQm2YD7yp48Jc142jjpZ2Lt1/9/nAW8gCIr/0yCq0qvH
VlGC7n3b8/4ydH80Q+n2CikRx13UHxMw3puSRaYXyjsUPg==
-----END CERTIFICATE-----
---
Server certificate
subject=/C=CN/ST=Zhangjiakou/L=Hebei/O=k8s/OU=System/CN=kubernetes
issuer=/C=CN/ST=Zhangjiakou/L=Hebei/O=k8s/OU=System/CN=kubernetes
---
No client certificate CA names sent
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 1547 bytes and written 326 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 93105081B4F2180D60C410055FAB1FA358848D230B2893D3211DB820CCAFF4A1
    Session-ID-ctx:
    Master-Key: FD7AF4DF7190B3E44C56FC19655DF5260C2A0450303A62EA91A5A171ED9388DF817B59D09CDB363735918DBFDBAA9475
    Start Time: 1592905789
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---
closed

It seems the ca.pem is OK, right?

spring2maz commented 4 years ago

Hi. This means your certificates should work fine.

In your listed files, I only find ca certificate and server certificate (no client certificate). If it's not intended for clients to provide a certificate, you should configure emqx with listener.ssl.external.verify = verify_none.

On the client side, it's worth trying the --insecure flag, see if it can connect without timeout.

--- Although it seems to me that the idle_timeout happens after TLS handshake on the server side. i.e emqx broker believed that TLS handshake is complete.

chunzhenzyd commented 4 years ago

Hi @spring2maz

  1. I uses one-way authentication. With setting listener.ssl.external.verify = verify_none, when connecting, an error poped: Error: read ECONNRESET in MQTTX, and no output in emqx erlang console;
  2. I uses MQTTX as the client, there is no --insecure option to be set. Would you please share the way to test with --insecure flag?
spring2maz commented 4 years ago

Hi. @chunzhenzyd In your previous reply, you mentioned that you tried testing with mosquitto to find difference between container/non-container. mosquitto has --insecure flag, please give it a try again. If --insecure works fine, try with --cafile flag instead.

For MQTTX, it seems there is no --insecure equivalent option. If mosquitto works fine with --cafile but MQTTX does not, then it's something we can follow up here: https://github.com/emqx/MQTTX/issues

chunzhenzyd commented 4 years ago

Hi @spring2maz

When testing mosquitto which is non-container, I also used MQTTX to connect...

BR

chunzhenzyd commented 4 years ago

Hi @spring2maz

Sorry to make misunderstanding.. When testing mosquitto, I mean I use MQTTX as the client, mosquitto(non-container) as the server.

BR

spring2maz commented 4 years ago

ah. I see. except for giving mosquitto client a try. I can not think of anything more to try. l will try to reproduce it.

spring2maz commented 4 years ago

Hi @chunzhenzyd

Sorry for the late reply. I have tested it locally, however failed to reproduce the idle_timeout error.

The script to run EMQX in docker:

#!/bin/bash

set -euo pipefail

THISDIR="$(cd $(dirname $0) && pwd)"

docker run -d -u root --restart=always --name mqtt --network host \
  --log-driver=json-file \
  --log-opt max-size=20m \
  --log-opt max-file=5 \
  -e EMQX_NAME=mqtt \
  -e EMQX_ZONE__EXTERNAL__MAX_MQUEUE_LEN=0 \
  -e EMQX_ZONE__INTERNAL__MAX_MQUEUE_LEN=0 \
  -e EMQX_MQTT__RETAIN_AVAILABLE=false \
  -e EMQX_ALLOW_ANONYMOUS=false \
  -e EMQX_ZONE__INTERNAL__ALLOW_ANONYMOUS=false \
  -e EMQX_LISTENER__SSL__EXTERNAL__KEYFILE=etc/certs/server-key.pem \
  -e EMQX_LISTENER__SSL__EXTERNAL__CERTFILE=etc/certs/server.pem \
  -e EMQX_LISTENER__SSL__EXTERNAL__CACERTFILE=etc/certs/ca.pem \
  -e EMQX_LOG__CONSOLE__LEVEL=debug \
  -v $THISDIR/certs:/opt/emqx/etc/certs \
  -v /etc/localtime:/etc/localtime:ro \
  emqx/emqx:latest

In MQTTX GUI, I checked "self-signed", Picked CA file path/name, and did not enable strict validation.

A list of thing which might be different comparing to your tests:

As I mentioned, idle_timeout is the server side did not receive connect message in time. this implies that the server believes TLS handshake has completed successfully. It would be helpful if you could capture the packets between MQTTX and EMQX to see:

  1. if TLS handshake is complete from both sides
  2. if any application data is sent after TLS handshake
chunzhenzyd commented 4 years ago

Hi @spring2maz

Thanks for your help! I guess "Host network is used for docker container, so the certificate is signed for host IP." is the most possible reason. Now we use nginx + certificates + 1883, I will close this case now, and will reopen it if there is a requirement. Thanks again!

BR