search

tendrilinc / marathon-autoscaler

A simple autoscaler for Marathon applications
https://hub.docker.com/r/tendril/marathon-autoscaler/
Apache License 2.0
41 stars 16 forks source link

Connection refused when starting #3

Closed cookandy closed 8 years ago

cookandy commented 8 years ago

Hello,

Thank you for the project! I am having some trouble getting the container started correctly. First, it seems like parameters like DATADOG_API_KEY must be set, even though we won't be using Datadog.

Next, I'm also noticing that the script is trying to go out to my Mesos slaves on port 5050 for some reason. Here's how I started the container:

docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker:/var/log/docker -e MESOS_URI=http://node1:5050 -e MARATHON_URI=http://node1:8080 -e MARATHON_USER=user -e MARATHON_PASS=pass -e DATADOG_API_KEY=a -e DATADOG_APP_KEY=a -e DATADOG_ENV=a -e INTERVAL=30 myregistry/marathon_autoscaler

However, when it starts, I see calls to node2 and node3 which are my 2 slaves:

/usr/lib/python2.7/site-packages/supervisor/options.py:296: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
  'Supervisord is running as root and it is searching '
2016-06-08 23:16:57,351 CRIT Supervisor running as root (no user in config file)
2016-06-08 23:16:57,351 WARN Included extra file "/etc/supervisor.d/marathon_autoscaler.ini" during parsing
2016-06-08 23:16:57,365 INFO RPC interface 'supervisor' initialized
2016-06-08 23:16:57,365 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2016-06-08 23:16:57,366 INFO supervisord started with pid 1
2016-06-08 23:16:58,370 INFO spawned: 'marathon_autoscaler' with pid 8
2016-06-08 23:16:58,622 | INFO | Namespace(cpu_fan_out=None, datadog_api_key='a', datadog_app_key='a', datadog_env='a', enforce_version_match=False, log_config='/app/logging_config.json', marathon_pass='pass', marathon_uri='http://node1:8080', marathon_user='user', mesos_uri='http://node1:5050', rules_prefix='mas_rule', sleep_interval=5)  (<module>:91)
2016-06-08 23:16:58,630 | INFO | Mesos and Marathon Connections Established.  (start:152)
2016-06-08 23:16:58,682 | ERROR | HTTPConnectionPool(host='node2', port=5050): Max retries exceeded with url: /monitor/statistics (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fdf63351750>: Failed to establish a new connection: [Errno 111] Connection refused',))  (_do_request:69)
2016-06-08 23:16:58,683 | ERROR | HTTPConnectionPool(host='node3', port=5050): Max retries exceeded with url: /monitor/statistics (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fdf63351690>: Failed to establish a new connection: [Errno 111] Connection refused',))  (_do_request:69)
2016-06-08 23:16:59,685 INFO success: marathon_autoscaler entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2016-06-08 23:17:01,710 | ERROR | HTTPConnectionPool(host='node2', port=5050): Max retries exceeded with url: /monitor/statistics (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fdf63350e10>: Failed to establish a new connection: [Errno 111] Connection refused',))  (_do_request:69)
2016-06-08 23:17:01,711 | ERROR | HTTPConnectionPool(host='node3', port=5050): Max retries exceeded with url: /monitor/statistics (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fdf63350d90>: Failed to establish a new connection: [Errno 111] Connection refused',))  (_do_request:69)
2016-06-08 23:17:04,715 | INFO | Stats differentials collected.  (poll:81)
2016-06-08 23:17:04,716 | ERROR | "'points' parameter is required"  (send_datadog_metrics:77)
2016-06-08 23:17:04,716 | INFO | Decision process beginning.  (decide:56)
2016-06-08 23:17:04,716 | INFO | Decisions are completed.  (start:160)

Any ideas?

kernelpanek-segfault commented 8 years ago

Hey @cookandy, we are aware of these problems and are going to open issues for them. We completely forgot that we put Mesos Agents on a different port, so we'll be making that configurable. The same thing goes for Datadog, we need to make it optional and eventually move it into a plugin-esque architecture so we can quickly add other monitoring solutions.