telefonicaid / rabbit-topic-acl

RabbitMQ Plugin for adding ACL-based security for topics
GNU Affero General Public License v3.0
3 stars 3 forks source link

paho client make crash rabbitmq with mqtt #39

Open AlvaroVega opened 5 years ago

AlvaroVega commented 5 years ago

Some simple variations of the same paho mqtt client makes crash rabbitmq (tested from 3.7.21, 3.8,1, 3.8.2rc versions):

docker-compose:

iot-rabbit:
  container_name: iot-rabbit
  hostname: iot-rabbit
  image:  rabbitmq:3.8.1
  ports:
    - "1883:1883"
    - "5672:5672"
    - "15672:15672"
  environment:
    - RABBITMQ_DEFAULT_USER=iota
    - RABBITMQ_DEFAULT_PASS=XXXXXX
  log_driver: json-file
  log_opt:
    max-size: "250m"
``

Test case:
```python
import paho.mqtt.client as mqtt

TOPIC = '/9n4hb1vpwbjozzmw9f0flf9c2/test2/attrs'
BROKER = "localhost"

client = mqtt.Client(
    # client_id="",
    # clean_session=True,
    # protocol=mqtt.MQTTv311,
)
client.username_pw_set('iota', 'XXXX')
client.connect(host=BROKER, port=1883)#, keepalive=60)
client.publish(TOPIC, str('{"v": 55}'), qos=0, retain=False)
#client.loop(10)
#client.disconnect();

rabbit crash report:

iot-rabbit                  | 2019-11-13 06:56:28.457 [error] <0.1301.0> ** Generic server <0.1301.0> terminating
iot-rabbit                  | ** Last message in was {tcp,#Port<0.73>,<<16,29,0,4,77,81,84,84,4,194,0,60,0,0,0,4,105,111,116,97,0,9,99,104,52,110,103,51,33,77,69,48,49,0,38,47,57,110,52,104,98,49,118,112,119,98,106,111,122,122,109,119,57,102,48,102,108,102,57,99,50,47,116,101,115,116,50,47,97,116,116,114,115,123,34,118,34,58,32,53,53,125>>}
iot-rabbit                  | ** When Server state == {state,#Port<0.73>,"172.17.0.1:33662 -> 172.17.0.3:1883",true,undefined,false,running,{none,none},<0.1300.0>,false,none,{proc_state,#Port<0.73>,#{},{undefined,undefined},{0,nil},{0,nil},undefined,1,undefined,undefined,undefined,{undefined,undefined},undefined,<<"amq.topic">>,{amqp_adapter_info,{0,0,0,0,0,65535,44049,3},1883,{0,0,0,0,0,65535,44049,1},33662,<<"172.17.0.1:33662 -> 172.17.0.3:1883">>,{'MQTT',"N/A"},[{channels,1},{channel_max,1},{frame_max,0},{client_properties,[{<<"product">>,longstr,<<"MQTT client">>}]},{ssl,false}]},none,undefined,undefined,#Fun<rabbit_mqtt_processor.0.88837742>,{0,0,0,0,0,65535,44049,1},#Fun<rabbit_mqtt_util.4.71278299>,#Fun<rabbit_mqtt_util.5.71278299>},undefined,{state,fine,5000,undefined}}
iot-rabbit                  | ** Reason for termination == 
iot-rabbit                  | ** {{badmatch,{error,einval}},[{rabbit_mqtt_processor,process_login,4,[{file,"src/rabbit_mqtt_processor.erl"},{line,551}]},{rabbit_mqtt_processor,process_request,3,[{file,"src/rabbit_mqtt_processor.erl"},{line,136}]},{rabbit_mqtt_processor,process_frame,2,[{file,"src/rabbit_mqtt_processor.erl"},{line,76}]},{rabbit_mqtt_reader,process_received_bytes,2,[{file,"src/rabbit_mqtt_reader.erl"},{line,291}]},{gen_server2,handle_msg,2,[{file,"src/gen_server2.erl"},{line,1067}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}
iot-rabbit                  | 2019-11-13 06:56:28.458 [error] <0.1301.0> CRASH REPORT Process <0.1301.0> with 0 neighbours exited with reason: no match of right hand value {error,einval} in rabbit_mqtt_processor:process_login/4 line 551 in gen_server2:terminate/3 line 1183
iot-rabbit                  | 2019-11-13 06:56:28.458 [error] <0.1299.0> Supervisor {<0.1299.0>,rabbit_mqtt_connection_sup} had child rabbit_mqtt_reader started with rabbit_mqtt_reader:start_link(<0.1300.0>, {acceptor,{0,0,0,0,0,0,0,0},1883}) at <0.1301.0> exit with reason no match of right hand value {error,einval} in rabbit_mqtt_processor:process_login/4 line 551 in context child_terminated
iot-rabbit                  | 2019-11-13 06:56:28.458 [error] <0.1299.0> Supervisor {<0.1299.0>,rabbit_mqtt_connection_sup} had child rabbit_mqtt_reader started with rabbit_mqtt_reader:start_link(<0.1300.0>, {acceptor,{0,0,0,0,0,0,0,0},1883}) at <0.1301.0> exit with reason reached_max_restart_intensity in context shutdown
jzgh commented 4 years ago

Hi, we were just evaluating using rabbitmq and hit a similar issue (discovered when running a load test and noticed very slow performance).

We have a custom light weight MQTT 3.1.1 client running in embedded FreeRTOS (can compile and run on linux with the same results).

Simple case is connect -> send 1 message -> close connection (after a random few connections -> crash)

For the load test, we created a pool of 10 connections, opened them all first, and started passing messages. Within a few seconds we get a failure.

rabbitmq 3.7.14 running on opensuse leap 15.1 (rabbitmq-server-3.7.14-lp151.1.9.x86_64.rpm) uname -a Linux kmu 4.12.14-lp151.28.48-default #1 SMP Fri Apr 17 05:38:36 UTC 2020 (18849d1) x86_64 x86_64 x86_64 GNU/Linux

2020-06-28 10:37:37.518 [error] <0.5305.0> Generic server <0.5305.0> terminating Last message in was {tcp,#Port<0.32175>,<<16,42,0,4,77,81,84,84,4,194,0,20,0,8,116,97,103,103,119,48,48,55,0,13,109,97,114,115,111,102,116,58,116,97,103,103,119,0,5,116,97,103,103,119>>} When Server state == {state,#Port<0.32175>,"192.168.1.16:54702 -> 192.168.1.13:1883",true,undefined,false,running,{none,none},<0.5304.0>,false,none,{proc_state,#Port<0.32175>,#{},{undefined,undefined},{0,nil},{0,nil},undefined,1,undefined,undefined,undefined,{undefined,undefined},undefined,<<"tags">>,{amqp_adapter_info,{0,0,0,0,0,65535,49320,269},1883,{0,0,0,0,0,65535,49320,272},54702,<<"192.168.1.16:54702 -> 192.168.1.13:1883">>,{'MQTT',"N/A"},[{channels,1},{channel_max,1},{frame_max,0},{client_properties,[{<<"product">>,longstr,<<"MQTT client">>}]},{ssl,false}]},none,undefined,undefined,#Fun,{0,0,0,0,0,65535,49320,272}},undefined,{state,fine,5000,undefined}} Reason for termination == ** {{badmatch,{error,einval}},[{rabbit_mqtt_processor,process_login,4,[{file,"src/rabbit_mqtt_processor.erl"},{line,521}]},{rabbit_mqtt_processor,process_request,3,[{file,"src/rabbit_mqtt_processor.erl"},{line,132}]},{rabbit_mqtt_processor,process_frame,2,[{file,"src/rabbit_mqtt_processor.erl"},{line,72}]},{rabbit_mqtt_reader,process_received_bytes,2,[{file,"src/rabbit_mqtt_reader.erl"},{line,282}]},{gen_server2,handle_msg,2,[{file,"src/gen_server2.erl"},{line,1056}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,247}]}]} 2020-06-28 10:37:37.520 [error] <0.5305.0> CRASH REPORT Process <0.5305.0> with 0 neighbours exited with reason: no match of right hand value {error,einval} in rabbit_mqtt_processor:process_login/4 line 521 in gen_server2:terminate/3 line 1172 2020-06-28 10:37:37.523 [error] <0.5303.0> Supervisor {<0.5303.0>,rabbit_mqtt_connection_sup} had child rabbit_mqtt_reader started with rabbit_mqtt_reader:start_link(<0.5304.0>, {acceptor,{0,0,0,0,0,0,0,0},1883}) at <0.5305.0> exit with reason no match of right hand value {error,einval} in rabbit_mqtt_processor:process_login/4 line 521 in context child_terminated 2020-06-28 10:37:37.526 [error] <0.5303.0> Supervisor {<0.5303.0>,rabbit_mqtt_connection_sup} had child rabbit_mqtt_reader started with rabbit_mqtt_reader:start_link(<0.5304.0>, {acceptor,{0,0,0,0,0,0,0,0},1883}) at <0.5305.0> exit with reason reached_max_restart_intensity in context shutdown