/var/log/* is much more than a bunch of forgotten files

[pjanuario]

Elasticsearch have a bunch of awesome free products, among them ELK stack: Elasticsearch, Logstash and Kibana.

This talk will get you on a walk to a centralised logging repository with analytics and powerful search. When you have several systems and machines, tracking and searching in the logs can be quite painful, well it was, because after this talk you don't have any excuse to keep suffering with this.

A small glimpse from what you can easily achieve in a couple of hours.

The Nginx access default format will allow you to have:

• number of requests on time (as the image shows);
• number of errors on time (as the image shows);
• number of requests split by response code;
• number of bytes;
• Top client ips;
• user agent info (browser, os, device);
• world map (using clien ips);

Adding the some data to logs you can easily get:

• requests metrics per upstream;
• upstream response times;
• metric per domain;

[pierreozoux]

:+1:

[joaojeronimo]

Hey this is a really really cool talk! I got that ELB stack together once a couple of months ago but it only ran for a week, after that it became horribly slow and we threw everything out the window and started using LogEntries because we should focus on what's important for us. But I would love to know more about this and what I did wrong.

[pjanuario]

@joaojeronimo it's actually weird! If you want feel free to get in touch with me, I have it currently running in production, for now just for our load balancer logs and soon with the all applications logs also.

[foliveira]

@pjanuario would you be up to do this talk at the end of June/start of July? (Basically next require(lx))

[pjanuario]

@foliveira Count me in!!