Incorrect config keys documented for management timeouts

[LHCGreg]

I'm running 3.7.17 using the rabbitmq:3.7.17 docker image.

I found that contrary to the documentation at https://www.rabbitmq.com/management.html, management.tcp.request_timeout, management.tcp.idle_timeout, and management.tcp.inactivity_timeout have to be management.listener.server.request_timeout, management.listener.server.idle_timeout, and management.listener.server.inactivity_timeout instead. If I try to use management.tcp.request_timeout, etc, Rabbit fails to start with the error:

2019-09-12 15:13:29.595 [debug] <0.490.0> Starting HTTP[S] listener with transport ranch_tcp, options [{port,15672}] and protocol options #{idle_timeout => 120000,inactivity_timeout => 120000,request_timeout => 120000,sendfile => false}, stream handlers [rabbit_cowboy_stream_h,cowboy_compress_h,cowboy_stream_h]
2019-09-12 15:13:29.596 [error] <0.490.0> ** Generic server rabbit_web_dispatch_registry terminating
** Last message in was {add,rabbitmq_management_tcp,[{cowboy_opts,[{idle_timeout,120000},{inactivity_timeout,120000},{request_timeout,120000},{sendfile,false}]},{port,15672}],#Fun<rabbit_web_dispatch.0.72790530>,[{'_',[],[{[],[],cowboy_static,{priv_file,rabbitmq_management,"www/index.html"}},{[<<"api">>,<<"overview">>],[],rabbit_mgmt_wm_overview,[]},{[<<"api">>,<<"cluster-name">>],[],rabbit_mgmt_wm_cluster_name,[]},{[<<"api">>,<<"nodes">>],[],rabbit_mgmt_wm_nodes,[]},{[<<"api">>,<<"nodes">>,node],[],rabbit_mgmt_wm_node,[]},{[<<"api">>,<<"nodes">>,node,<<"memory">>],[],rabbit_mgmt_wm_node_memory,[absolute]},{[<<"api">>,<<"nodes">>,node,<<"memory">>,<<"relative">>],[],rabbit_mgmt_wm_node_memory,[relative]},{[<<"api">>,<<"nodes">>,node,<<"memory">>,<<"ets">>],[],rabbit_mgmt_wm_node_memory_ets,...},...]}],...}
** When Server state == undefined
** Reason for termination ==
** {{incompatible_listeners,{"RabbitMQ Management",[{cowboy_opts,[{idle_timeout,120000},{inactivity_timeout,120000},{request_timeout,120000},{sendfile,false}]},{port,15672}]},{"RabbitMQ Management",[{cowboy_opts,[{sendfile,false}]},{port,15672},{ssl,false}]}},[{rabbit_web_dispatch_registry,handle_call,3,[{file,"src/rabbit_web_dispatch_registry.erl"},{line,92}]},{gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,661}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,690}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}

I only knew to use management.listener.server.request_timeout instead because of the helpful "did you mean ..." log message when there's a wrong config setting.

[michaelklishin]

Please provide a way to reproduce when making such claims.

Both are supported but management.tcp.* and management.ssl.* are the more consistent, easier to remember way available as of 3.7.9. Legacy options support was retained for backwards compatibility.

[michaelklishin]

• Test snippets for the above config schema keys
• The following one line config file used by a node with management plugin enabled

  management.tcp.port = 15672

  is effective and can be verified using rabbitmq-diagnostics environment and rabbitmq-diagnostics listeners: Interface: [::], port: 15678, protocol: http, purpose: HTTP API

My only guess is that you are not actually running 3.7.17 but rather something before 3.7.9.

[LHCGreg]

The output of rabbitmqctl status inside the container is:

Status of node rabbit@rabbitmq-dev-0.rabbitmq-dev-discovery-service.rabbitmq-dev.svc.cluster.local ...
[{pid,499},
 {running_applications,
     [{rabbitmq_management,"RabbitMQ Management Console","3.7.17"},
      {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.7.17"},
      {cowboy,"Small, fast, modern HTTP server.","2.6.1"},
      {cowlib,"Support library for manipulating Web protocols.","2.7.0"},
      {rabbitmq_management_agent,"RabbitMQ Management Agent","3.7.17"},
      {amqp_client,"RabbitMQ AMQP Client","3.7.17"},
      {rabbitmq_peer_discovery_k8s,
          "Kubernetes-based RabbitMQ peer discovery backend","3.7.17"},
      {rabbitmq_peer_discovery_common,
          "Modules shared by various peer discovery backends","3.7.17"},
      {rabbit,"RabbitMQ","3.7.17"},
      {rabbit_common,
          "Modules shared by rabbitmq-server and rabbitmq-erlang-client",
          "3.7.17"},
      {ranch,"Socket acceptor pool for TCP protocols.","1.7.1"},
      {ssl,"Erlang/OTP SSL application","9.3.5"},
      {public_key,"Public key infrastructure","1.6.7"},
      {asn1,"The Erlang ASN1 compiler version 5.0.9","5.0.9"},
      {mnesia,"MNESIA  CXC 138 12","4.16"},
      {stdout_formatter,
          "Tools to format paragraphs, lists and tables as plain text",
          "0.2.2"},
      {jsx,"a streaming, evented json parsing toolkit","2.9.0"},
      {observer_cli,"Visualize Erlang Nodes On The Command Line","1.5.0"},
      {xmerl,"XML parser","1.3.21"},
      {crypto,"CRYPTO","4.5.1"},
      {inets,"INETS  CXC 138 49","7.0.9"},
      {sysmon_handler,"Rate-limiting system_monitor event handler","1.1.0"},
      {os_mon,"CPO  CXC 138 46","2.5"},
      {recon,"Diagnostic tools for production use","2.5.0"},
      {lager,"Erlang logging framework","3.6.10"},
      {goldrush,"Erlang event stream processor","0.1.9"},
      {compiler,"ERTS  CXC 138 10","7.4.4"},
      {syntax_tools,"Syntax tools","2.2"},
      {sasl,"SASL  CXC 138 11","3.4"},
      {stdlib,"ERTS  CXC 138 10","3.9.2"},
      {kernel,"ERTS  CXC 138 10","6.4.1"}]},
 {os,{unix,linux}},
 {erlang_version,
     "Erlang/OTP 22 [erts-10.4.4] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:64]\n"},
 {memory,
     [{connection_readers,118584},
      {connection_writers,59732},
      {connection_channels,318544},
      {connection_other,498628},
      {queue_procs,708284},
      {queue_slave_procs,0},
      {plugins,2832248},
      {other_proc,23318528},
      {metrics,250764},
      {mgmt_db,1052984},
      {mnesia,121952},
      {other_ets,2939952},
      {binary,2809464},
      {msg_index,34064},
      {code,27517311},
      {atom,1254625},
      {other_system,12315776},
      {allocated_unused,19101040},
      {reserved_unallocated,0},
      {strategy,rss},
      {total,[{erlang,76151440},{rss,92155904},{allocated,95252480}]}]},
 {alarms,[]},
 {listeners,[{clustering,25672,"::"},{amqp,5672,"::"},{http,15672,"::"}]},
 {vm_memory_calculation_strategy,rss},
 {vm_memory_high_watermark,0.4},
 {vm_memory_limit,858993459},
 {disk_free_limit,50000000},
 {disk_free,5107138560},
 {file_descriptors,
     [{total_limit,65436},
      {total_used,10},
      {sockets_limit,58890},
      {sockets_used,4}]},
 {processes,[{limit,1048576},{used,480}]},
 {run_queue,1},
 {uptime,8536},
 {kernel,{net_ticktime,60}}]

So either it is 3.7.17, or it somehow thinks it is 3.7.17 but is not. I'll put together some steps for reproducing.

[LHCGreg]

ok, so if I run the 3.7.17 docker image with no environment variables, with a one line config of management.tcp.request_timeout = 122000, everything works fine. But (with the same config) if I set environment variables RABBITMQ_DEFAULT_USER and RABBITMQ_DEFAULT_PASS, the error happens.

To reproduce, save this as Dockerfile:

FROM rabbitmq:3.7.17

RUN echo "management.tcp.request_timeout = 122000" > /etc/rabbitmq/rabbitmq.conf
RUN echo "[rabbitmq_management]." > /etc/rabbitmq/enabled_plugins
EXPOSE 15672

Then run

docker build -t lhcgreg_repro .
docker run -it -p 9999:15672 --env "RABBITMQ_DEFAULT_USER=rabbitmq" --env "RABBITMQ_DEFAULT_PASS=rabbitmq_password" lhcgreg_repro

It looks like RABBITMQ_DEFAULT_USER and RABBITMQ_DEFAULT_PASS are understood by the docker image rather than RabbitMQ itself. Maybe the docker image is doing something funky. I'll take this up with the maintainers of the docker image.

[michaelklishin]

How to Verify Effective Configuration

[michaelklishin]

Available Environment Variables do not include RABBITMQ_DEFAULT_USER. I highly recommend configuring as much as possible using RabbitMQ config file. The tendency to use environment variables heavily in the Docker image is a design choice I would likely never understand.

[lukebakken]

It looks like RABBITMQ_DEFAULT_USER and RABBITMQ_DEFAULT_PASS are understood by the docker image rather than RabbitMQ itself

Those values are probably used to create invalid configuration for RabbitMQ.