Document how to add a configuration file

[spuder]

Once you have the docker container up and running, you will get the following errors in the logs.

core@core-01 ~ $ docker logs elk
2015-01-14 22:20:13,821 CRIT Supervisor running as root (no user in config file)
2015-01-14 22:20:13,821 WARN Included extra file "/etc/supervisor/conf.d/supervisord.conf" during parsing
2015-01-14 22:20:13,843 INFO RPC interface 'supervisor' initialized
2015-01-14 22:20:13,844 WARN cElementTree not installed, using slower XML parser for XML-RPC
2015-01-14 22:20:13,844 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2015-01-14 22:20:13,844 INFO supervisord started with pid 1
2015-01-14 22:20:14,847 INFO spawned: 'nginx' with pid 9
2015-01-14 22:20:14,848 INFO spawned: 'elasticsearch' with pid 10
2015-01-14 22:20:14,850 INFO spawned: 'logstash' with pid 11
2015-01-14 22:20:15,852 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-01-14 22:20:15,856 INFO success: elasticsearch entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-01-14 22:20:15,856 INFO success: logstash entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-01-14 22:20:29,089 INFO exited: logstash (exit status 1; not expected)
2015-01-14 22:20:30,090 INFO spawned: 'logstash' with pid 78
2015-01-14 22:20:31,093 INFO success: logstash entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-01-14 22:20:40,455 INFO exited: logstash (exit status 1; not expected)
2015-01-14 22:20:41,457 INFO spawned: 'logstash' with pid 103
2015-01-14 22:20:42,460 INFO success: logstash entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-01-14 22:20:51,883 INFO exited: logstash (exit status 1; not expected)
2015-01-14 22:20:52,887 INFO spawned: 'logstash' with pid 127

I assume that this is because a config file has not been made. It would be nice if the documentation explained how to add a config file to the volumes

[erichelgeson]

Just had the same experience, new to the ELK stack so not sure where to get started.

[vjm]

+1

Any ideas on why logstash keeps exiting?

[blacktop]

Hi all! Sorry for the delay.

So I will start to debug the logstash and see what I can do. However, this was essentially an experiment for me and in practice you normally wouldn't want to run all your services in one container. It's great for demos though :wink:

So what you want is to use docker-compose and link a few Elasticsearch containers to an Kibana/Nginx to a logstash container and then use logstash-forwarder to send your logs to the ELK stack.

I am interested in making general purpose building blocks that can be easily combined and I am still trying to figure out the best way to do that.

Right now if you use the kibana4 branch or do a docker pull blacktop/elk:4 you can run it with:

$ docker run -it --name elk4 -p 80:80 -p 9200:9200 blacktop/elk:4 

Then you can index directly to it with something like elasticsearch-py or something and see it in kibana.

Kibana 4 is very beautiful and very 'Splunk-like' however, you can't get passed the settings screen until it has an index to parse. Just FYI. So start it like I mentioned and then do something like:

https://elasticsearch-py.readthedocs.org/en/master/

from datetime import datetime
from elasticsearch import Elasticsearch
es = Elasticsearch(['http://<enter boot2docker ip or ip assigned to docker container here>'])

for i in range(10000):
    doc = {
        'author': 'kimchy',
        'text': 'Elasticsearch: cool. bonsai cool.',
        'timestamp': datetime.now()
    }
    res = es.index(index="test-index", doc_type='tweet', id=i, body=doc)
    # print(res['created'])

res = es.get(index="test-index", doc_type='tweet', id=1)
print(res['_source'])

es.indices.refresh(index="test-index")

res = es.search(index="test-index", body={"query": {"match_all": {}}})
print("Got %d Hits:" % res['hits']['total'])
for hit in res['hits']['hits']:
    print("%(timestamp)s %(author)s: %(text)s" % hit["_source"])

Now navigate to the boot2docker ip or docker ip in a web browser. You will be prompted for a user/pass which defaults to user: admin, password: admin. Now enter * in the index field and select timestamp then you can go to the Discover tab and see those absolutely gorgeous logs!