Runlevel 1

[ratmav]

Once #1 is complete, we need to be able to use Ansible to drive Docker to create Elasticsearch and Logstash containers. The images should be available via DockerHub at https://hub.docker.com/r/pblittle.

[ratmav]

We're far enough along with #1 that I can start on this. The Ansible Docker module should be very useful here.

[ratmav]

I wrote a playbook and role to pull the requested Logstash image, but I wasn't able to test it on my local VM's due to an expired SSL certificate error from https://get.docker.com (I was re-running the Docker installation playbook on fresh VM's). I logged a support request w/ Docker to see what's going on.

@GiorgioRegni: I noticed is that P.B. LIttle's Elasticsearch image is just the official image with some AWS plugins. Is this the image we want to use?

[ratmav]

We only need https://hub.docker.com/r/pblittle/docker-logstash/.

[ratmav]

I've got this going on CentOS 7, and Ubuntu 14.04 should work, but needs testing.

[ratmav]

The expired certificate error is on my end - the VM's I'm using for local testing are having intermittent networking issues. If this continues, I'll take another approach.

[ratmav]

The SSL issue on my end was due to the TimeOffset property being enabled by default on my VM's. At any rate, the runlevel 1 playbook builds and runs the Logstash container - tested on Ubuntu 14.04 and CentOS 7.

[ratmav]

Ok, so spoke with @GiorgioRegni yesterday about spinning up an Elasticsearch cluster now that we can install Docker, build containers, and start the containers. On a high level, my understanding is that we want a cluster of Elasticsearch containers talking to each other for log plundering, and each container would also have a running Logstash, etc. instance so that the we UI is available in multiple places.

That said, I'm new to Elasticsearch, although I've had some experience with Solr. My first step is to research ElasticSearch setup, etc. and look at clustering on standard VM's, then we can start moving forward with containerizing the cluster.

[ratmav]

Found the Elasticsearch Guide, which I'm going to spend some time with to get a better understanding of the tool.

[ratmav]

I've done some reasearch on working with the ELK stack and Docker. I don't have much information in what or how we're logging things, but I assume that the application just needs to be able to make REST calls to what looks like it's going to be a logging microservice being shipped as an appliance. Along with the Elasticsearch guide, I found some interesting resources that would help with prototyping:

• https://hub.docker.com/r/itzg/elasticsearch/
• http://nathanleclaire.com/blog/2015/04/27/automating-docker-logging-elasticsearch-logstash-kibana-and-logspout/

At this point it would be useful to have a running version of the application (or at least some example GETs and PUTs I can loop in a script), so that I could try building a Logstash cluster running on two VM's, both running Docker with a single container as cluster nodes. @GiorgioRegni: Does it matter at all what we actually log at this point? If not I'll just use some dummy JSON and increment a timestamp.

[ratmav]

I've been thinking about how to break the Logstash cluster up, and I think this is going to require Docker's "ambassador" pattern again.

Last time I looked at this socat was an issue to due network performance. I don't think that is the case here, since this is logging. I need to confirm that, but at least it looks like a way forward.

[ratmav]

deviantony/docker-elk also looks useful, but I wonder if it makes more sense to run all of the ELK stack in a single container here. That's probably an anti-pattern, though.

I just got some feedback and redirection from @GiorgioRegni, and the architecture for logging is a single ELK deploy per Docker (storage) host. The repo listed above is probably a good way to go in that case then. However, we do want to have the ElasticSearch container from each Docker host (storage node) for a cluster across all Docker hosts.

[ratmav]

I think I've been overthinking this a great deal - apparently you just have to configure a YAML file with a cluster name for Elasticsearch to discover itself over the network. So, it might be worth forking that ELK stack or just using it's Dockerfiles as a starting point.

[GiorgioRegni]

Yes, expose is just fine and for test, you can just use a dummy json with a timestamp for now