RabbitMQ (and thus VOLTTRON) does not start with log error messages such as:
2019-10-30 22:49:27.064 [error] <0.49.0> Loading of /home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/lib/rabbitmq/mnesia/rabbit@XXXXlion-plugins-expand/rabbitmq_amqp1_0-3.7.7/ebin/rabbit_amqp1_0_util.beam failed: badfile 2019-10-30 22:49:27.064 [error] emulator beam/beam_load.c(2314): Error loading function rabbit_amqp1_0_util:serial_diff/2: op i_plus q x p x:
no specific operation found
In diagnosing the issue, I found that the documentation specifies incorrect Erlang Version Compatibility for RabbitMQ 3.7.7. The Install Erlang step specifies versions 21 or above. However the RabbitMQ documentation specifies that 21.3.X is the max.
Installing the correct version of Erlang fixes the issue.
Affected Version
6.0 through develop
Screenshots
See gists below for output.
Expected
Volttron starts with usual methods.
Actual
./start-volttron displays:
Starting VOLTTRON verbosely in the background with VOLTTRON_HOME=/home/volttron/.volttron
Waiting for VOLTTRON to startup..
VOLTTRON startup failed/timed out. Please check volttron.log for details
Bootstrap with --rabbitmq (Not sure this is necessary)
./start-volttron, ./start/rabbitmq or equivalent
Additional Details
Issue encountered on Raspbian Stretch.
rabbitmq.log
2019-10-30 22:49:17.086 [info] <0.8.0> Log file opened with Lager
2019-10-30 22:49:25.191 [info] <0.313.0>
Starting RabbitMQ 3.7.7 on Erlang 22.0
Copyright (C) 2007-2018 Pivotal Software, Inc.
Licensed under the MPL. See http://www.rabbitmq.com/
2019-10-30 22:49:25.195 [info] <0.313.0>
node : rabbit@XXXXlion
home dir : /home/volttron
config file(s) : /home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/etc/rabbitmq/rabbitmq.conf
cookie hash : RgrxocHsFe1SffjvUiVQYQ==
log(s) : /home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/log/rabbitmq/rabbit@XXXXlion.log
: /home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/log/rabbitmq/rabbit@XXXXlion_upgrade.log
database dir : /home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/lib/rabbitmq/mnesia/rabbit@XXXXlion
2019-10-30 22:49:27.064 [error] <0.49.0> Loading of /home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/lib/rabbitmq/mnesia/rabbit@XXXXlion-plugins-expand/rabbitmq_amqp1_0-3.7.7/ebin/rabbit_amqp1_0_util.beam failed: badfile 2019-10-30 22:49:27.064 [error] emulator beam/beam_load.c(2314): Error loading function rabbit_amqp1_0_util:serial_diff/2: op i_plus q x p x:
no specific operation found
2019-10-30 22:49:28.206 [warning] <0.319.0> You are using a 32-bit version of Erlang: you may run into memory address~nspace exhaustion or statistic counters overflow.~n
2019-10-30 22:49:28.208 [info] <0.319.0> Memory high watermark set to 370 MiB (388425318 bytes) of 926 MiB (971063296 bytes) total
2019-10-30 22:49:28.220 [info] <0.321.0> Enabling free disk space monitoring
2019-10-30 22:49:28.221 [info] <0.321.0> Disk free limit set to 50MB
2019-10-30 22:49:28.232 [info] <0.323.0> Limiting to approx 1048476 file handles (943626 sockets)
2019-10-30 22:49:28.232 [info] <0.324.0> FHC read buffering: OFF
2019-10-30 22:49:28.232 [info] <0.324.0> FHC write buffering: ON
2019-10-30 22:49:28.236 [info] <0.313.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2019-10-30 22:49:28.350 [error] <0.49.0> Loading of /home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/lib/rabbitmq/mnesia/rabbit@XXXXlion-plugins-expand/rabbitmq_amqp1_0-3.7.7/ebin/rabbit_amqp1_0_util.beam failed: badfile 2019-10-30 22:49:28.350 [error] emulator beam/beam_load.c(2314): Error loading function rabbit_amqp1_0_util:serial_diff/2: op i_plus q x p x:
no specific operation found
2019-10-30 22:49:28.430 [error] <0.49.0> Loading of /home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/lib/rabbitmq/mnesia/rabbit@XXXXlion-plugins-expand/rabbitmq_amqp1_0-3.7.7/ebin/rabbit_amqp1_0_util.beam failed: badfile 2019-10-30 22:49:28.431 [error] emulator beam/beam_load.c(2314): Error loading function rabbit_amqp1_0_util:serial_diff/2: op i_plus q x p x:
no specific operation found
2019-10-30 22:49:28.461 [info] <0.313.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2019-10-30 22:49:28.461 [info] <0.313.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping registration.
2019-10-30 22:49:28.465 [info] <0.313.0> Priority queues enabled, real BQ is rabbit_variable_queue
2019-10-30 22:49:28.516 [info] <0.346.0> Starting rabbit_node_monitor
2019-10-30 22:49:28.570 [error] <0.49.0> Loading of /home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/lib/rabbitmq/mnesia/rabbit@XXXXlion-plugins-expand/rabbitmq_amqp1_0-3.7.7/ebin/rabbit_amqp1_0_util.beam failed: badfile 2019-10-30 22:49:28.571 [error] emulator beam/beam_load.c(2314): Error loading function rabbit_amqp1_0_util:serial_diff/2: op i_plus q x p x:
no specific operation found
2019-10-30 22:49:28.653 [error] <0.49.0> Loading of /home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/lib/rabbitmq/mnesia/rabbit@XXXXlion-plugins-expand/rabbitmq_amqp1_0-3.7.7/ebin/rabbit_amqp1_0_util.beam failed: badfile 2019-10-30 22:49:28.654 [error] emulator beam/beam_load.c(2314): Error loading function rabbit_amqp1_0_util:serial_diff/2: op i_plus q x p x:
no specific operation found
2019-10-30 22:49:28.698 [info] <0.313.0> Management plugin: using rates mode 'basic'
2019-10-30 22:49:28.703 [info] <0.386.0> Making sure data directory '/home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/lib/rabbitmq/mnesia/rabbit@XXXXlion/msg_stores/vhosts/7D0517K09K0L24TSZVKJJ94TG' for vhost 'volttron' exists
2019-10-30 22:49:28.738 [info] <0.386.0> Starting message stores for vhost 'volttron'
2019-10-30 22:49:28.739 [info] <0.390.0> Message store "7D0517K09K0L24TSZVKJJ94TG/msg_store_transient": using rabbit_msg_store_ets_index to provide index
2019-10-30 22:49:28.744 [info] <0.386.0> Started message store of type transient for vhost 'volttron'
2019-10-30 22:49:28.746 [info] <0.393.0> Message store "7D0517K09K0L24TSZVKJJ94TG/msg_store_persistent": using rabbit_msg_store_ets_index to provide index
2019-10-30 22:49:28.754 [info] <0.386.0> Started message store of type persistent for vhost 'volttron'
2019-10-30 22:49:28.879 [info] <0.443.0> Making sure data directory '/home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/lib/rabbitmq/mnesia/rabbit@XXXXlion/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists
2019-10-30 22:49:28.924 [info] <0.443.0> Starting message stores for vhost '/'
2019-10-30 22:49:28.926 [info] <0.447.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index
2019-10-30 22:49:28.932 [info] <0.443.0> Started message store of type transient for vhost '/'
2019-10-30 22:49:28.933 [info] <0.450.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index
2019-10-30 22:49:28.941 [info] <0.443.0> Started message store of type persistent for vhost '/'
2019-10-30 22:49:28.976 [info] <0.508.0> started TCP Listener on [::]:5672
2019-10-30 22:49:28.982 [info] <0.524.0> started SSL Listener on [::]:5671
2019-10-30 22:49:28.984 [info] <0.313.0> Setting up a table for connection tracking on this node: tracked_connection_on_node_rabbit@XXXXlion
2019-10-30 22:49:28.985 [info] <0.313.0> Setting up a table for per-vhost connection counting on this node: tracked_connection_per_vhost_on_node_rabbit@XXXXlion
2019-10-30 22:49:29.069 [error] <0.49.0> Loading of /home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/lib/rabbitmq/mnesia/rabbit@XXXXlion-plugins-expand/rabbitmq_amqp1_0-3.7.7/ebin/rabbit_amqp1_0_util.beam failed: badfile 2019-10-30 22:49:29.070 [error] emulator beam/beam_load.c(2314): Error loading function rabbit_amqp1_0_util:serial_diff/2: op i_plus q x p x:
no specific operation found
2019-10-30 22:49:29.151 [error] <0.49.0> Loading of /home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/lib/rabbitmq/mnesia/rabbit@XXXXlion-plugins-expand/rabbitmq_amqp1_0-3.7.7/ebin/rabbit_amqp1_0_util.beam failed: badfile 2019-10-30 22:49:29.152 [error] emulator beam/beam_load.c(2314): Error loading function rabbit_amqp1_0_util:serial_diff/2: op i_plus q x p x:
no specific operation found
2019-10-30 22:49:29.195 [info] <0.613.0> Management plugin started. Port: 15671
2019-10-30 22:49:29.196 [info] <0.720.0> Statistics database started.
2019-10-30 22:49:29.205 [notice] <0.117.0> Changed loghwm of /home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/log/rabbitmq/rabbit@XXXXlion.log to 50
2019-10-30 22:49:29.646 [info] <0.8.0> Server startup complete; 10 plugins started.
* rabbitmq_shovel_management
* rabbitmq_federation_management
* rabbitmq_management
* rabbitmq_shovel
* rabbitmq_auth_mechanism_ssl
* rabbitmq_web_dispatch
* rabbitmq_management_agent
* rabbitmq_trust_store
* rabbitmq_amqp1_0
* rabbitmq_federation
2019-10-30 22:49:32.206 [warning] <0.417.0> Federation exchange 'volttron' in vhost 'volttron' did not connect to exchange 'volttron' in vhost 'volttron' on amqps://bluelion:5671/volttron
{error,ehostunreach}
2019-10-30 22:49:32.214 [error] <0.416.0> Supervisor {<0.416.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],
<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',
none,<<"upstream-volttron-bluelion">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/c...">>],...},...}) at <0.417.0> exit with reason {shutdown,restart} in context child_terminated
2019-10-30 22:49:35.326 [warning] <0.741.0> Federation exchange 'volttron' in vhost 'volttron' did not connect to exchange 'volttron' in vhost 'volttron' on amqps://bluelion:5671/volttron
{error,ehostunreach}
2019-10-30 22:49:35.330 [error] <0.416.0> Supervisor {<0.416.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],
<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',
none,<<"upstream-volttron-bluelion">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/c...">>],...},...}) at <0.741.0> exit with reason {shutdown,restart} in context child_terminated
2019-10-30 22:49:43.486 [warning] <0.762.0> Federation exchange 'volttron' in vhost 'volttron' did not connect to exchange 'volttron' in vhost 'volttron' on amqps://bluelion:5671/volttron
{error,ehostunreach}
2019-10-30 22:49:43.490 [error] <0.416.0> Supervisor {<0.416.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],
<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',
none,<<"upstream-volttron-bluelion">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/c...">>],...},...}) at <0.762.0> exit with reason {shutdown,restart} in context child_terminated
2019-10-30 22:49:51.646 [warning] <0.782.0> Federation exchange 'volttron' in vhost 'volttron' did not connect to exchange 'volttron' in vhost 'volttron' on amqps://bluelion:5671/volttron
{error,ehostunreach}
2019-10-30 22:49:51.650 [error] <0.416.0> Supervisor {<0.416.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],
<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',
none,<<"upstream-volttron-bluelion">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/c...">>],...},...}) at <0.782.0> exit with reason {shutdown,restart} in context child_terminated
2019-10-30 22:49:59.806 [warning] <0.802.0> Federation exchange 'volttron' in vhost 'volttron' did not connect to exchange 'volttron' in vhost 'volttron' on amqps://bluelion:5671/volttron
{error,ehostunreach}
2019-10-30 22:49:59.810 [error] <0.416.0> Supervisor {<0.416.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],
<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',
none,<<"upstream-volttron-bluelion">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/c...">>],...},...}) at <0.802.0> exit with reason {shutdown,restart} in context child_terminated
2019-10-30 22:50:07.966 [warning] <0.822.0> Federation exchange 'volttron' in vhost 'volttron' did not connect to exchange 'volttron' in vhost 'volttron' on amqps://bluelion:5671/volttron
{error,ehostunreach}
2019-10-30 22:50:07.970 [error] <0.416.0> Supervisor {<0.416.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],
<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',
none,<<"upstream-volttron-bluelion">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/c...">>],...},...}) at <0.822.0> exit with reason {shutdown,restart} in context child_terminated
2019-10-30 22:50:16.126 [warning] <0.842.0> Federation exchange 'volttron' in vhost 'volttron' did not connect to exchange 'volttron' in vhost 'volttron' on amqps://bluelion:5671/volttron
{error,ehostunreach}
2019-10-30 22:50:16.131 [error] <0.416.0> Supervisor {<0.416.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],
<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',
none,<<"upstream-volttron-bluelion">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/c...">>],...},...}) at <0.842.0> exit with reason {shutdown,restart} in context child_terminated
2019-10-30 22:50:24.365 [warning] <0.862.0> Federation exchange 'volttron' in vhost 'volttron' did not connect to exchange 'volttron' in vhost 'volttron' on amqps://bluelion:5671/volttron
{error,ehostunreach}
2019-10-30 22:50:24.370 [error] <0.416.0> Supervisor {<0.416.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],
<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',
none,<<"upstream-volttron-bluelion">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/c...">>],...},...}) at <0.862.0> exit with reason {shutdown,restart} in context child_terminated
2019-10-30 22:50:32.526 [warning] <0.882.0> Federation exchange 'volttron' in vhost 'volttron' did not connect to exchange 'volttron' in vhost 'volttron' on amqps://bluelion:5671/volttron
{error,ehostunreach}
2019-10-30 22:50:32.530 [error] <0.416.0> Supervisor {<0.416.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],
<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',
none,<<"upstream-volttron-bluelion">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/c...">>],...},...}) at <0.882.0> exit with reason {shutdown,restart} in context child_terminated
2019-10-30 22:50:40.686 [warning] <0.902.0> Federation exchange 'volttron' in vhost 'volttron' did not connect to exchange 'volttron' in vhost 'volttron' on amqps://bluelion:5671/volttron
{error,ehostunreach}
2019-10-30 22:50:40.690 [error] <0.416.0> Supervisor {<0.416.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],
<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',
none,<<"upstream-volttron-bluelion">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/c...">>],...},...}) at <0.902.0> exit with reason {shutdown,restart} in context child_terminated
2019-10-30 22:50:48.846 [warning] <0.922.0> Federation exchange 'volttron' in vhost 'volttron' did not connect to exchange 'volttron' in vhost 'volttron' on amqps://bluelion:5671/volttron
{error,ehostunreach}
2019-10-30 22:50:48.850 [error] <0.416.0> Supervisor {<0.416.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],
<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',
none,<<"upstream-volttron-bluelion">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/c...">>],...},...}) at <0.922.0> exit with reason {shutdown,restart} in context child_terminated
2019-10-30 22:50:57.006 [warning] <0.942.0> Federation exchange 'volttron' in vhost 'volttron' did not connect to exchange 'volttron' in vhost 'volttron' on amqps://bluelion:5671/volttron
{error,ehostunreach}
2019-10-30 22:50:57.010 [error] <0.416.0> Supervisor {<0.416.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],
<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',
none,<<"upstream-volttron-bluelion">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/c...">>],...},...}) at <0.942.0> exit with reason {shutdown,restart} in context child_terminated
2019-10-30 22:51:05.246 [warning] <0.962.0> Federation exchange 'volttron' in vhost 'volttron' did not connect to exchange 'volttron' in vhost 'volttron' on amqps://bluelion:5671/volttron
{error,ehostunreach}
2019-10-30 22:51:05.250 [error] <0.416.0> Supervisor {<0.416.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],
<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',
none,<<"upstream-volttron-bluelion">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/c...">>],...},...}) at <0.962.0> exit with reason {shutdown,restart} in context child_terminated
2019-10-30 22:51:13.406 [warning] <0.982.0> Federation exchange 'volttron' in vhost 'volttron' did not connect to exchange 'volttron' in vhost 'volttron' on amqps://bluelion:5671/volttron
{error,ehostunreach}
2019-10-30 22:51:13.410 [error] <0.416.0> Supervisor {<0.416.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],
<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',
none,<<"upstream-volttron-bluelion">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/c...">>],...},...}) at <0.982.0> exit with reason {shutdown,restart} in context child_terminated
2019-10-30 22:51:21.566 [warning] <0.1002.0> Federation exchange 'volttron' in vhost 'volttron' did not connect to exchange 'volttron' in vhost 'volttron' on amqps://bluelion:5671/volttron
{error,ehostunreach}
2019-10-30 22:51:21.570 [error] <0.416.0> Supervisor {<0.416.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],
<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',
none,<<"upstream-volttron-bluelion">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/c...">>],...},...}) at <0.1002.0> exit with reason {shutdown,restart} in context child_terminated
2019-10-30 22:51:29.805 [warning] <0.1023.0> Federation exchange 'volttron' in vhost 'volttron' did not connect to exchange 'volttron' in vhost 'volttron' on amqps://bluelion:5671/volttron
{error,ehostunreach}
2019-10-30 22:51:29.807 [error] <0.416.0> Supervisor {<0.416.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],
<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',
none,<<"upstream-volttron-bluelion">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/c...">>],...},...}) at <0.1023.0> exit with reason {shutdown,restart} in context child_terminated
2019-10-30 22:51:37.966 [warning] <0.1043.0> Federation exchange 'volttron' in vhost 'volttron' did not connect to exchange 'volttron' in vhost 'volttron' on amqps://bluelion:5671/volttron
{error,ehostunreach}
2019-10-30 22:51:37.970 [error] <0.416.0> Supervisor {<0.416.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],
<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',
none,<<"upstream-volttron-bluelion">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/c...">>],...},...}) at <0.1043.0> exit with reason {shutdown,restart} in context child_terminated
2019-10-30 22:51:46.206 [warning] <0.1063.0> Federation exchange 'volttron' in vhost 'volttron' did not connect to exchange 'volttron' in vhost 'volttron' on amqps://bluelion:5671/volttron
{error,ehostunreach}
2019-10-30 22:51:46.210 [error] <0.416.0> Supervisor {<0.416.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],
<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',
none,<<"upstream-volttron-bluelion">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/c...">>],...},...}) at <0.1063.0> exit with reason {shutdown,restart} in context child_terminated
2019-10-30 22:51:54.446 [warning] <0.1083.0> Federation exchange 'volttron' in vhost 'volttron' did not connect to exchange 'volttron' in vhost 'volttron' on amqps://bluelion:5671/volttron
{error,ehostunreach}
2019-10-30 22:51:54.451 [error] <0.416.0> Supervisor {<0.416.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],
<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',
none,<<"upstream-volttron-bluelion">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/c...">>],...},...}) at <0.1083.0> exit with reason {shutdown,restart} in context child_terminated
2019-10-30 22:52:01.376 [info] <0.1123.0> RabbitMQ is asked to stop...
2019-10-30 22:52:01.633 [info] <0.1123.0> Stopping RabbitMQ applications and their dependencies in the following order:
rabbitmq_shovel_management
rabbitmq_federation_management
rabbitmq_management
rabbitmq_shovel
rabbitmq_amqp1_0
rabbitmq_federation
amqp_client
rabbitmq_web_dispatch
cowboy
cowlib
rabbitmq_auth_mechanism_ssl
rabbitmq_management_agent
rabbitmq_trust_store
rabbit
mnesia
rabbit_common
os_mon
amqp10_client
amqp10_common
2019-10-30 22:52:01.633 [info] <0.1123.0> Stopping application 'rabbitmq_shovel_management'
2019-10-30 22:52:01.634 [info] <0.43.0> Application rabbitmq_shovel_management exited with reason: stopped
2019-10-30 22:52:01.634 [info] <0.1123.0> Stopping application 'rabbitmq_federation_management'
2019-10-30 22:52:01.634 [info] <0.43.0> Application rabbitmq_federation_management exited with reason: stopped
2019-10-30 22:52:01.634 [info] <0.1123.0> Stopping application 'rabbitmq_management'
2019-10-30 22:52:01.650 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped
2019-10-30 22:52:01.651 [info] <0.1123.0> Stopping application 'rabbitmq_shovel'
2019-10-30 22:52:01.661 [info] <0.43.0> Application rabbitmq_shovel exited with reason: stopped
2019-10-30 22:52:01.661 [info] <0.1123.0> Stopping application 'rabbitmq_amqp1_0'
2019-10-30 22:52:01.661 [info] <0.43.0> Application rabbitmq_amqp1_0 exited with reason: stopped
2019-10-30 22:52:01.661 [info] <0.1123.0> Stopping application 'rabbitmq_federation'
2019-10-30 22:52:01.671 [info] <0.43.0> Application rabbitmq_federation exited with reason: stopped
2019-10-30 22:52:01.671 [info] <0.1123.0> Stopping application 'amqp_client'
2019-10-30 22:52:01.675 [error] <0.1103.0> ** Generic server <0.1103.0> terminating
** Last message in was {'$gen_cast',maybe_go}
** When Server state == {not_started,{{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',none,<<"upstream-volttron-bluelion">>,false},{upstream_params,<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>,{amqp_params_network,<<"guest">>,<<"guest">>,<<"volttron">>,"bluelion",5671,2047,0,10,60000,[{server_name_indication,"bluelion"},{fail_if_no_peer_cert,true},{verify,verify_peer},{keyfile,"/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem"},{certfile,"/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt"},{cacertfile,"/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt"}],[#Fun<amqp_uri.12.90191702>],[],[]},{exchange,{resource,<<"volttron">>,exchange,<<"volttron">>},topic,true,false,false,[{<<"alternate-exchange">>,longstr,<<"undeliverable">>}],[{federation,[{{<<"upstream-volttron-bluelion">>,<<"volttron">>},<<"A">>}]}],[{vhost,<<"volttron">>},{name,<<"volttron-federation">>},{pattern,<<"^volttron">>},{'apply-to',<<"exchanges">>},{definition,[{<<"federation-upstream-set">>,<<"all">>}]},{priority,0}],undefined,{[],[rabbit_federation_exchange]},#{user => <<"guest">>}},<<"amqps://bluelion:5671/volttron">>,[{<<"uri">>,longstr,<<"amqps://bluelion:5671/volttron">>},{<<"exchange">>,longstr,...}]},...}}
** Reason for termination ==
** {killed,{gen_server,call,[<0.1119.0>,connect,60000]}}
2019-10-30 22:52:01.676 [error] <0.1103.0> CRASH REPORT Process <0.1103.0> with 0 neighbours exited with reason: {killed,{gen_server,call,[<0.1119.0>,connect,60000]}} in gen_server2:terminate/3 line 1166
2019-10-30 22:52:01.678 [error] <0.416.0> Supervisor {<0.416.0>,rabbit_federation_link_sup} had child {upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt&certfile=/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt&keyfile=/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem&verify=verify_peer&fail_if_no_peer_cert=true&auth_mechanism=external&server_name_indication=bluelion">>],
<<"volttron">>,<<"volttron">>,1000,1,5,none,none,false,'on-confirm',
none,<<"upstream-volttron-bluelion">>,false} started with rabbit_federation_exchange_link:start_link({{upstream,[<<"amqps://bluelion:5671/volttron?cacertfile=/home/volttron/.volttron/certificates/c...">>],...},...}) at <0.1103.0> exit with reason {killed,{gen_server,call,[<0.1119.0>,connect,60000]}} in context child_terminated
2019-10-30 22:52:01.678 [warning] <0.1113.0> Channel (<0.1113.0>): Unregistering confirm handler <0.1103.0> because it died. Reason: {killed,{gen_server,call,[<0.1119.0>,connect,60000]}}
2019-10-30 22:52:01.686 [info] <0.43.0> Application amqp_client exited with reason: stopped
2019-10-30 22:52:01.686 [info] <0.1123.0> Stopping application 'rabbitmq_web_dispatch'
2019-10-30 22:52:01.696 [info] <0.1123.0> Stopping application 'cowboy'
2019-10-30 22:52:01.696 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped
2019-10-30 22:52:01.706 [info] <0.43.0> Application cowboy exited with reason: stopped
2019-10-30 22:52:01.706 [info] <0.1123.0> Stopping application 'cowlib'
2019-10-30 22:52:01.706 [info] <0.43.0> Application cowlib exited with reason: stopped
2019-10-30 22:52:01.706 [info] <0.1123.0> Stopping application 'rabbitmq_auth_mechanism_ssl'
2019-10-30 22:52:01.717 [info] <0.43.0> Application rabbitmq_auth_mechanism_ssl exited with reason: stopped
2019-10-30 22:52:01.716 [info] <0.1123.0> Stopping application 'rabbitmq_management_agent'
2019-10-30 22:52:01.728 [info] <0.1123.0> Stopping application 'rabbitmq_trust_store'
2019-10-30 22:52:01.728 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped
2019-10-30 22:52:01.739 [info] <0.1123.0> Stopping application 'rabbit'
2019-10-30 22:52:01.739 [info] <0.43.0> Application rabbitmq_trust_store exited with reason: stopped
2019-10-30 22:52:01.739 [info] <0.313.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping unregistration.
2019-10-30 22:52:01.740 [info] <0.524.0> stopped SSL Listener on [::]:5671
2019-10-30 22:52:01.742 [info] <0.508.0> stopped TCP Listener on [::]:5672
2019-10-30 22:52:01.744 [info] <0.340.0> Closing all connections in vhost 'volttron' on node 'rabbit@XXXXlion' because the vhost is stopping
2019-10-30 22:52:01.744 [info] <0.450.0> Stopping message store for directory '/home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/lib/rabbitmq/mnesia/rabbit@XXXXlion/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent'
2019-10-30 22:52:01.744 [info] <0.340.0> Closing all connections in vhost '/' on node 'rabbit@XXXXlion' because the vhost is stopping
2019-10-30 22:52:01.811 [info] <0.393.0> Stopping message store for directory '/home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/lib/rabbitmq/mnesia/rabbit@XXXXlion/msg_stores/vhosts/7D0517K09K0L24TSZVKJJ94TG/msg_store_persistent'
2019-10-30 22:52:01.840 [info] <0.450.0> Message store for directory '/home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/lib/rabbitmq/mnesia/rabbit@XXXXlion/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped
2019-10-30 22:52:01.841 [info] <0.447.0> Stopping message store for directory '/home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/lib/rabbitmq/mnesia/rabbit@XXXXlion/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient'
2019-10-30 22:52:01.881 [info] <0.393.0> Message store for directory '/home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/lib/rabbitmq/mnesia/rabbit@XXXXlion/msg_stores/vhosts/7D0517K09K0L24TSZVKJJ94TG/msg_store_persistent' is stopped
2019-10-30 22:52:01.882 [info] <0.390.0> Stopping message store for directory '/home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/lib/rabbitmq/mnesia/rabbit@XXXXlion/msg_stores/vhosts/7D0517K09K0L24TSZVKJJ94TG/msg_store_transient'
2019-10-30 22:52:01.910 [info] <0.447.0> Message store for directory '/home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/lib/rabbitmq/mnesia/rabbit@XXXXlion/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped
2019-10-30 22:52:01.950 [info] <0.390.0> Message store for directory '/home/volttron/rabbitmq_server/rabbitmq_server-3.7.7/var/lib/rabbitmq/mnesia/rabbit@XXXXlion/msg_stores/vhosts/7D0517K09K0L24TSZVKJJ94TG/msg_store_transient' is stopped
2019-10-30 22:52:02.015 [info] <0.43.0> Application rabbit exited with reason: stopped
2019-10-30 22:52:02.015 [info] <0.1123.0> Stopping application 'mnesia'
2019-10-30 22:52:02.028 [info] <0.43.0> Application mnesia exited with reason: stopped
2019-10-30 22:52:02.028 [info] <0.1123.0> Stopping application 'rabbit_common'
2019-10-30 22:52:02.028 [info] <0.1123.0> Stopping application 'os_mon'
2019-10-30 22:52:02.028 [info] <0.43.0> Application rabbit_common exited with reason: stopped
2019-10-30 22:52:02.038 [info] <0.1123.0> Stopping application 'amqp10_client'
2019-10-30 22:52:02.038 [info] <0.43.0> Application os_mon exited with reason: stopped
2019-10-30 22:52:02.047 [info] <0.43.0> Application amqp10_client exited with reason: stopped
2019-10-30 22:52:02.047 [info] <0.1123.0> Stopping application 'amqp10_common'
2019-10-30 22:52:02.048 [info] <0.43.0> Application amqp10_common exited with reason: stopped
2019-10-30 22:52:02.048 [info] <0.1123.0> Successfully stopped RabbitMQ and its dependencies
2019-10-30 22:52:02.048 [info] <0.1123.0> Halting Erlang VM with the following applications:
ranch_proxy_protocol
ranch
ssl
public_key
asn1
xmerl
crypto
jsx
recon
inets
lager
goldrush
compiler
syntax_tools
syslog
sasl
stdlib
kernel
volttron.log
2019-10-30 22:49:05,012 () volttron.platform.main DEBUG: VOLTTRON PLATFORM RUNNING ON rmq MESSAGEBUS
2019-10-30 22:49:05,012 () volttron.platform.main DEBUG: ********************************************************************
2019-10-30 22:49:07,231 () rmq_setup.pyc DEBUG: Rabbitmq is not running. Attempting to start
2019-10-30 22:49:30,739 () rmq_setup.pyc INFO: Rmq server at /home/volttron/rabbitmq_server/rabbitmq_server-3.7.7 is running at
2019-10-30 22:49:30,740 () root DEBUG: Creating RMQ Core config.store
2019-10-30 22:49:30,742 () volttron.platform.vip.agent.core DEBUG: address: inproc://vip
2019-10-30 22:49:30,743 () volttron.platform.vip.agent.core DEBUG: identity: config.store
2019-10-30 22:49:30,743 () volttron.platform.vip.agent.core DEBUG: agent_uuid: None
2019-10-30 22:49:30,745 () volttron.platform.vip.agent.core DEBUG: serverkey: None
2019-10-30 22:49:30,747 () volttron.platform.vip.agent.core DEBUG: AGENT RUNNING on RMQ Core xxxxlion.config.store
2019-10-30 22:49:30,894 () volttron.utils.rmq_mgmt DEBUG: Exception when trying to make HTTP request to https://xxxxlion:15671/api/users/ with args {'headers': {'Content-Type': 'application/json'}, 'cert': ('/home/volttron/.volttron/certificates/certs/xxxxlion-admin.crt', '/home/volttron/.volttron/certificates/private/xxxxlion-admin.pem'), 'auth': ('xxxxlion-admin', 'default_passwd'), 'verify': '/home/volttron/.volttron/certificates/certs/xxxxlion-trusted-cas.crt'} : 'NoneType' object has no attribute 'raise_for_status'
2019-10-30 22:49:30,895 () volttron.platform.main ERROR: Unhandled exception in rmq router loop
Traceback (most recent call last):
File "/home/volttron/volttron/volttron/platform/main.py", line 764, in rmq_router
bind_web_address=opts.bind_web_address
File "/home/volttron/volttron/volttron/platform/vip/rmq_router.py", line 114, in __init__
param = self._build_connection_parameters()
File "/home/volttron/volttron/volttron/platform/vip/rmq_router.py", line 127, in _build_connection_parameters
self._instance_name)
File "/home/volttron/volttron/volttron/utils/rmq_mgmt.py", line 996, in build_router_connection
self.create_user_with_permissions(rmq_user, permissions, ssl_auth=self.is_ssl)
File "/home/volttron/volttron/volttron/utils/rmq_mgmt.py", line 907, in create_user_with_permissions
if user not in self.get_users(ssl_auth):
File "/home/volttron/volttron/volttron/utils/rmq_mgmt.py", line 255, in get_users
response = self._http_get_request(url, ssl_auth)
File "/home/volttron/volttron/volttron/utils/rmq_mgmt.py", line 170, in _http_get_request
response = self._call_grequest('get', url, ssl_auth)
File "/home/volttron/volttron/volttron/utils/rmq_mgmt.py", line 123, in _call_grequest
raise e
AttributeError: 'NoneType' object has no attribute 'raise_for_status'
2019-10-30 22:49:30,898 () volttron.platform.main DEBUG: In RMQ router finally
2019-10-30 22:49:30,911 () volttron.platform.main DEBUG: AIP finally
2019-10-30 22:49:30,912 () volttron.platform.main DEBUG: ********************************************************************
2019-10-30 22:49:30,913 () volttron.platform.main DEBUG: VOLTTRON PLATFORM HAS SHUTDOWN
2019-10-30 22:49:30,913 () volttron.platform.main DEBUG: ********************************************************************
Description of Issue
RabbitMQ (and thus VOLTTRON) does not start with log error messages such as:
In diagnosing the issue, I found that the documentation specifies incorrect Erlang Version Compatibility for RabbitMQ 3.7.7. The Install Erlang step specifies versions 21 or above. However the RabbitMQ documentation specifies that
21.3.X
is the max.Installing the correct version of Erlang fixes the issue.
Affected Version
6.0 through develop
Screenshots
See gists below for output.
Expected
Volttron starts with usual methods.
Actual
./start-volttron
displays:And log output below.
Steps to Reproduce
--rabbitmq
(Not sure this is necessary)./start-volttron
,./start/rabbitmq
or equivalentAdditional Details
Issue encountered on Raspbian Stretch.
rabbitmq.log
volttron.log