search

octohost / www

Main website for the octohost project.
http://www.octohost.io/
Apache License 2.0
6 stars 4 forks source link

Document logspout and logstash usage #5

Open joshuacox opened 9 years ago

joshuacox commented 9 years ago

fairly self explanatory, I typed in octohost and logstash into the all knowing google and it returned this poor guy: https://groups.io/org/groupsio/octohost/message/177

we can do better than that. I'm going to experiment with this stuff this weekend. I'll do what I can to document things along the way. So far it looks like,

  1. docker pull logspout
  2. root@octo:~# octo logspout start Need to define REMOTE_SYSLOG_PORT in /etc/default/octohost
  3. vim /etc/default/octohost and edit REMOTE_SYSLOG_PORT
  4. build a VM with logstash on it to eat logs and name it after some famous beaver, this might be an appropriate task for packer.
darron commented 9 years ago

You're very close to exactly how I used it - but I ended up sending all of my logs to Papertrail. These are all of the logging related components:

https://gist.github.com/darron/98c97be93775dc831456

You need to update LOGSPOUT_TARGET, REMOTE_SYSLOG_PORT and REMOTE_SYSLOG_DESTINATION - then you get:

  1. octo logspout start - All container logs sent to LOGSPOUT_TARGET
  2. octo logs octostart - Use remote_syslog to send Docker and nginx logs to remote syslog.
  3. octo logs start {full-log-path} {mount} - Use remote_syslog to send {full-log-path} to remote syslog.

Most of that is in the octo help - but yes - could be better explained for sure.

joshuacox commented 9 years ago

So I started here: https://github.com/joshuacox/elkto but I quickly wanted to connect it to a newer elastic search: https://github.com/joshuacox/elasticto and Kibana 4: https://github.com/joshuacox/kibanocto

funny enough this all works great for external hosts, but neither logspout nor the remote syslog container want to connect to it when I specify the server's IP address. I also have a redis setup: https://github.com/joshuacox/elkto_datastore https://github.com/joshuacox/elkto_datastore_data

but I think the multiple container linking is borking on that. Oh well, it's working remarkably well right now for a test setup, data is even resilient inside elasticsearch, though I do still want to move that externally still. But for now it is a great testing setup. I could see spawning an ELK stack just to solve a problem across many hosts, and then pull the whole thing down afterwards, completely disposable.