search

sczyh30 / vertx-blueprint-microservice

Vert.x Blueprint Project - Micro-Shop microservice application
Apache License 2.0
772 stars 301 forks source link

Add ELK logging integration #3

Closed sczyh30 closed 7 years ago

sczyh30 commented 8 years ago

Add ELK logging integration in the microservice blueprint.

crazyfrozenpenguin commented 7 years ago

What do you want to use here?

Logstash-GELF is a great option here. What are you thinking in terms of logging libs? slf4j + log4j2 bridge are good options.

Thoughts?

sczyh30 commented 7 years ago

Yeah, SLF4J + Log4j2 can be a good choice as logging libs. As for ELK integration, there is a good example we can reference: Centralized logging for Vert.x applications using the ELK stack. But I'm not familiar with ELK stack... What do you think of the example compared with Logstash-GELF?

cescoffier commented 7 years ago

The appender approach is less and less used. It lacks of flexibility (ask settings for instance), and can be dangerous to use (blocking calls to a distributed system).

If you want to use ELK, you should avoid appenders and use FileBeat to read, parse and send your logs.

crazyfrozenpenguin commented 7 years ago

I've added an implementation using Vert.x SLF4J Log4J. Logstash GELF format appender used. This should suffice but I'll have a look into FileBeat.

crazyfrozenpenguin commented 7 years ago

Just converted log4j appenders to async appenders which should guarantee non-blocking solution.

In addition, I had a quick look into Elasticsearch FileBeat and apparently it requires a log file to "attach to" (as shown in @sczyh30 link). In other words, we will have to log to local file and then use FileBeat to send the logs to Logstash.

This might be a good alternative if external volumes for the log files are mounted to the docker container.

Thoughts?