search

grafana / loki

Like Prometheus, but for logs.
https://grafana.com/loki
GNU Affero General Public License v3.0
23.94k stars 3.45k forks source link

promtail Logback Appender #604

Closed RobertHeim closed 4 years ago

RobertHeim commented 5 years ago

Is your feature request related to a problem? Please describe. In order to integrate Loki with the Java and especially with the Spring Boot ecosystem we need promtail support for Java based loggers.

Describe the solution you'd like It would be great to have a Logback Appender for promtail contributed to https://github.com/sndyuk/logback-more-appenders

Describe alternatives you've considered There is a fluentd appender in the logback-more-appenders package that could be used in combination with the provided fluent-plugin-loki. However, it does not work due to a name conflict. I think we could fix this, but in compliance with the overall grafana strategy it could promote promtail if there is a dedicated logback appender.

Additional context (none)

steven-sheehy commented 5 years ago

I use loki/promtail with Spring and Logback and it works without a logback appender. If you're running in Kubernetes, just configure your application to log to stdout then promtail will scrape the log files on that node. Even if you're not running in Kubernetes, just log to a file and configure promtail to scrape that file.

RobertHeim commented 5 years ago

I see that promtail primarely targets these deployments (kubernetes, or file logging) but a (rather small and not critical) project running on heroku and the (junior) developers not using docker but just a fat jar would benefit from a logback appender to keep the tech stack small.

Or am I missing something on how to setup promtail to catch up on a spring log file within a heroku deployment? But even if there is a solution, I still suggest an appender to integrate with logback.

slim-bean commented 5 years ago

The challenges here would be incorporating the natively compiled promtail app into a java runtime, you'd need to start looking at JNI/JNA which is always treading into murky waters, especially not knowing (or controlling) what the underlying heroku runtime environment looks like.

A better solution might be writing a logback appender that pushes directly to the Loki API.

I think we can leave this issue open to get a feel for community interest, add your thumbs up to the original comment if this is something you are interested in.

stale[bot] commented 5 years ago

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

jcdauchy commented 5 years ago

Now that loki is GA. Would be awesome to have such appender.

roidelapluie commented 5 years ago

Hello, I think that once #1275 is merged you can just use the syslog logback appender :)

jcdauchy commented 4 years ago

I will try thanks. It might be approach.

slim-bean commented 4 years ago

I see the stalebot closed this issue, however I think there still would be value in a logback appender that can send directly to Loki

@adityacs it looks like you have some experience in Java and the work you started doing for the logstash plugin with jruby (minus the jruby party which we wouldn't need for a pure java solution) might be useful here?

Is this something you would be interested in working on also? I can provide some guidance on how we would like to setup the Java packaging and how to incorporate this into the project

adityacs commented 4 years ago

@slim-bean Definitely, I will look into this. Just to confirm we are speaking of this logback right https://github.com/qos-ch/logback ?

slim-bean commented 4 years ago

Yup that's the one!

On Thu, Mar 19, 2020 at 11:21 AM Aditya C S notifications@github.com wrote:

@slim-bean https://github.com/slim-bean Definitely, I will look into this. Just to confirm we are speaking of this logback right https://github.com/qos-ch/logback ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/grafana/loki/issues/604#issuecomment-601239470, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACO2RK4RJCAJKUMMIXWRMITRIIZ53ANCNFSM4HNYUUKQ .

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

wilerson commented 4 years ago

Out of curiosity, what's the state for #1908 ? I'm evaluating some logging solutions and having a way to use logback directly would be a big selling point towards Loki.

nehaev commented 4 years ago

Please check https://github.com/loki4j/loki-logback-appender

sm3sher commented 3 years ago

@nehaev unfortunately, this does not seem to work with Quarkus

nehaev commented 3 years ago

@sm3sher It should work with any Java app. Feel free to file a detailed issue here, I'll take a look.