search

eclipse-mosquitto / mosquitto

Eclipse Mosquitto - An open source MQTT broker
https://mosquitto.org
Other
9.07k stars 2.39k forks source link

Ubuntu 20.04 ipv6 disabled in grub, mosquitto 2.0.2, Websocket fails to come up #1955

Open argupta23 opened 3 years ago

argupta23 commented 3 years ago

I installed mosquitto 2.0.2 on Ubuntu 20.4 and worked around the other issues mentioned in other thread and was able to get mosquitto up and running.

I then decided to make a change to an IPv4 only environment by disabling IPv6 in grub.

GRUB_CMDLINE_LINUX_DEFAULT="maybe-ubiquity ipv6.disable=1"

did an update-grub and rebooted the system

Here is my mosquitto.conf file

listener 1883
protocol mqtt
listener 1884 172.20.20.36
protocol websockets

I have even tried removing the IP address from the listener 1884 line, with the same result

Here is the associated mosquitto.log 

1608087208: Config loaded from /etc/mosquitto/mosquitto.conf.
1608087208: Opening ipv4 listen socket on port 1883.
1608087208: Opening ipv6 listen socket on port 1883.
1608087208: Warning: Address family not supported by protocol
1608087208: Opening websockets listen socket on port 1884.
1608087208: Initial logging level 255
1608087208: Libwebsockets version: 3.2.0 unknown-build-hash
1608087208: IPV6 compiled in and enabled
1608087208:  LWS_DEF_HEADER_LEN    : 4096
1608087208:  LWS_MAX_PROTOCOLS     : 5
1608087208:  LWS_MAX_SMP           : 1
1608087208:  sizeof (*info)        : 672
1608087208:  SYSTEM_RANDOM_FILEPATH: '/dev/urandom'
1608087208:  HTTP2 support         : available
1608087208: _realloc: size 3712: context
1608087208: context created
1608087208: Using event loop: poll
1608087208: Default ALPN advertisment: h2,http/1.1
1608087208:  default timeout (secs): 20
1608087208: _realloc: size 4936: pt_serv_buf
1608087208:  Threads: 1 each 1024 fds
1608087208:  mem: context:          7808 B (3712 ctx + (1 thr x 4096))
1608087208:  mem: http hdr size:   (2048 + 976), max count 1024
1608087208: _realloc: size 8192: fds table
1608087208:  mem: pollfd map:       8192 B
1608087208: _realloc: size 8192: lws_lookup
1608087208:  mem: platform fd map:  8192 B
1608087208: _realloc: size 840: event pipe wsi
1608087208: lws_role_transition: 0x5584e8c65f00: wsistate 0x200, ops pipe
1608087208: event pipe fd 7
1608087208: __insert_wsi_socket_into_fds: 0x5584e8c65f00: tsi=0, sock=7, pos-in-fds=0
1608087208:  Compiled with OpenSSL support
1608087208: Doing SSL library init
1608087208: _realloc: size 816: create vhost
1608087208: _realloc: size 224: vhost-specific plugin table
1608087208: _realloc: size 72: same vh list
1608087208: Creating Vhost 'default' port 1884, 3 protocols, IPv6 on
1608087208:  Using non-SSL mode
1608087208: _realloc: size 72: client ctx tcr
1608087208: lws_tls_client_create_vhost_context: vh default: created new client ctx 0
1608087208: created client ssl context for default
1608087208:  interface lo vs 172.20.20.36 (fam 17) ipv6 1
1608087208:  interface ens160 vs 172.20.20.36 (fam 17) ipv6 1
1608087208:  interface lo vs 172.20.20.36 (fam 2) ipv6 1
1608087208:  interface ens160 vs 172.20.20.36 (fam 2) ipv6 1
1608087208: initial if check says 0
1608087208: ERROR opening socket
1608087208:  mem: per-conn:          840 bytes + protocol rx buf
1608087208:  canonical_hostname = mosquitto22
1608087208: lws_plat_drop_app_privileges: not changing group

checking on the ports with ss -tulpn results in the below

Netid       State        Recv-Q       Send-Q             Local Address:Port               Peer Address:Port       Process
udp         UNCONN       0            0                  127.0.0.53%lo:53                      0.0.0.0:*
tcp         LISTEN       0            4096               127.0.0.53%lo:53                      0.0.0.0:*
tcp         LISTEN       0            128                      0.0.0.0:22                      0.0.0.0:*
tcp         LISTEN       0            128                    127.0.0.1:6010                    0.0.0.0:*
tcp         LISTEN       0            100                      0.0.0.0:1883                    0.0.0.0:*

If I enable IPv6, I am able to get 1884 to work and ss -tulpn output looks like

Netid       State        Recv-Q       Send-Q             Local Address:Port               Peer Address:Port       Process
udp         UNCONN       0            0                  127.0.0.53%lo:53                      0.0.0.0:*
tcp         LISTEN       0            4096               127.0.0.53%lo:53                      0.0.0.0:*
tcp         LISTEN       0            128                      0.0.0.0:22                      0.0.0.0:*
tcp         LISTEN       0            100                0.0.0.0:1883                    0.0.0.0:*
tcp         LISTEN       0            128                         [::]:22                         [::]:*
tcp         LISTEN       0            128                            *:1884                          *:*

Ideally I would like to associate an IP address to the port, I know it has worked in the past. so wondering what changed.

Happy to answer any questions or provide more logs if needed.

Thanks

sectokia commented 3 years ago

What happens if you add 'socketdomain ipv4' for the listener?

karlp commented 3 years ago

fwiw, we reverted lws 3.2 in openwrt, after problems with it.

argupta23 commented 3 years ago

What happens if you add 'socketdomain ipv4' for the listener?

@sectokia Thanks for the feedback.

with IPv6 enabled in grub, I added to mosquitto.conf the line and restarted mosquitto

listeners 1884 subdomain ipv4 --> I assume that is what you meant

intrestingly, ss -tulpn results in the following

argupta@mosquitto22:~$ ss -tulpn
Netid     State      Recv-Q     Send-Q           Local Address:Port           Peer Address:Port     Process
udp       UNCONN     0          0                127.0.0.53%lo:53                  0.0.0.0:*
tcp       LISTEN     0          4096             127.0.0.53%lo:53                  0.0.0.0:*
tcp       LISTEN     0          128                    0.0.0.0:22                  0.0.0.0:*
tcp       LISTEN     0          128                  127.0.0.1:6010                0.0.0.0:*
tcp       LISTEN     0          100                    0.0.0.0:1883                0.0.0.0:*
tcp       LISTEN     0          128                       [::]:22                     [::]:*
tcp       LISTEN     0          128                      [::1]:6010                   [::]:*
tcp       LISTEN     0          100                       [::]:1883                   [::]:*

and the mosquitto.log shows the following

1608132600: mosquitto version 2.0.2 starting
1608132600: Config loaded from /etc/mosquitto/mosquitto.conf.
1608132600: Opening ipv4 listen socket on port 1883.
1608132600: Opening ipv6 listen socket on port 1883.
1608132600: Opening websockets listen socket on port 1884.
1608132600: Initial logging level 255
1608132600: Libwebsockets version: 3.2.0 unknown-build-hash
1608132600: IPV6 compiled in and enabled
1608132600:  LWS_DEF_HEADER_LEN    : 4096
1608132600:  LWS_MAX_PROTOCOLS     : 5
1608132600:  LWS_MAX_SMP           : 1
1608132600:  sizeof (*info)        : 672
1608132600:  SYSTEM_RANDOM_FILEPATH: '/dev/urandom'
1608132600:  HTTP2 support         : available
1608132600: _realloc: size 3712: context
1608132600: context created
1608132600: Using event loop: poll
1608132600: Default ALPN advertisment: h2,http/1.1
1608132600:  default timeout (secs): 20
1608132600: _realloc: size 4936: pt_serv_buf
1608132600:  Threads: 1 each 1024 fds
1608132600:  mem: context:          7808 B (3712 ctx + (1 thr x 4096))
1608132600:  mem: http hdr size:   (2048 + 976), max count 1024
1608132600: _realloc: size 8192: fds table
1608132600:  mem: pollfd map:       8192 B
1608132600: _realloc: size 8192: lws_lookup
1608132600:  mem: platform fd map:  8192 B
1608132600: _realloc: size 840: event pipe wsi
1608132600: lws_role_transition: 0x5592475f7520: wsistate 0x200, ops pipe
1608132600: event pipe fd 8
1608132600: __insert_wsi_socket_into_fds: 0x5592475f7520: tsi=0, sock=8, pos-in-fds=0
1608132600:  Compiled with OpenSSL support
1608132600: Doing SSL library init
1608132600: _realloc: size 816: create vhost
1608132600: _realloc: size 224: vhost-specific plugin table
1608132600: _realloc: size 72: same vh list
1608132600: Creating Vhost 'default' port 1884, 3 protocols, IPv6 on
1608132600:  Using non-SSL mode
1608132600: _realloc: size 72: client ctx tcr
1608132600: lws_tls_client_create_vhost_context: vh default: created new client ctx 0
1608132600: created client ssl context for default
1608132600:  interface lo vs subdomain (fam 17) ipv6 1
1608132600:  interface ens160 vs subdomain (fam 17) ipv6 1
1608132600:  interface lo vs subdomain (fam 2) ipv6 1
1608132600:  interface ens160 vs subdomain (fam 2) ipv6 1
1608132600:  interface lo vs subdomain (fam 10) ipv6 1
1608132600:  interface ens160 vs subdomain (fam 10) ipv6 1
1608132600: lws_socket_bind: netif subdomain: Does not exist
1608132600: initial if check says -1
1608132600: deferred iface: adding vh default
1608132600: VH default: iface subdomain port 1884 DOESN'T EXIST
1608132600:  mem: per-conn:          840 bytes + protocol rx buf
1608132600:  canonical_hostname = mosquitto22
1608132600: lws_plat_drop_app_privileges: not changing group
1608132600: lws_plat_drop_app_privileges: not changing user
1608132600: lws_cancel_service
1608132600: mosquitto version 2.0.2 running
1608132600: lws_protocol_init
argupta23 commented 3 years ago

fwiw, we reverted lws 3.2 in openwrt, after problems with it.

@karlp Can you please share some details around it?

within the logs, I see

Libwebsockets version: 3.2.0 unknown-build-hash

Thanks

karlp commented 3 years ago

it was broken? we reverted to 3.1?

ralight commented 3 years ago

@argupta23 try this configuration:

listener 1883
protocol mqtt
listener 1884 172.20.20.36
protocol websockets
socketdomain ipv4
ralight commented 3 years ago

@karlp There is a specific workaround for lws 3.2.0 since mosquitto 1.6.7.

argupta23 commented 3 years ago

@argupta23 try this configuration:

listener 1883
protocol mqtt
listener 1884 172.20.20.36
protocol websockets
socketdomain ipv4

@ralight @sectokia I think you meant to write "socket_domain " with that I am able to get websocket to comeup on with an IPv4 address. BTW I just tested this in 1.6.9.

Also it looks like websockets is broken in 2.0.2 when using libwesocket 3.2

@ralight I have two questions here

  1. can you please point me to the workaround that you mentioned above?
  2. Is there a 1.6.12 package available for 20.4 when not using SNAP? I could only find 1.6.9 / 2.0.2. I am happy to do a source compile if you can send me details on 1. Thanks
ralight commented 3 years ago

@argupta23 Sorry, the PPA only stores the most recent version for a given Ubuntu release as far as I am aware.

The workaround for lws 3.2 is in the code already, you shouldn't need to do anything. I suspect that the real problem here is that the lws version in 20.04 does not have the external poll support built in, so it will never work :(

It seems that I need to provide a version of lws that works, for the more recent Ubuntu PPA versions as well.

The snap version would work just fine today...

ralight commented 3 years ago

Is this still a problem?