Overriding logging properties like FILE_LOG_PATTERN

[marceloverdijk]

Feature request for adding support for overriding logging properties via application.properties/yaml.

E.g. I want to easily override the FILE_LOG_PATTERN property. To define this in the application.properties/yaml also gives the benefit that I can change it per environment.

Boot already offers

# LOGGING
logging.path=/var/logs
logging.file=myapp.log
logging.config= # location of config file (default classpath:logback.xml for logback)
logging.level.*= # levels for loggers, e.g. "logging.level.org.springframework=DEBUG" (TRACE, DEBUG, INFO, WARN, ERROR, FATAL, OFF)

Something like below maybe?

logging.property.FILE_LOG_PATTERN=

Reference: http://stackoverflow.com/questions/26629197/overriding-the-file-log-pattern-if-possible-per-env

[backs999]

This would be very useful as I also have just come across the need to override the pattern for our logging indexer +1

[joensson]

+1 - This would indeed be very useful. We have the same need - and would also like to set other properties for use in logback config in the same manner.

[khoubyari]

+1

[philwebb]

Seems like a popular request. I'll assign to 1.2 and take a look but we are really near GA so it might need to be pushed back to 1.3.

[marceloverdijk]

Since I raised this feature request I needed to log to 2 separate files. 1 standard log file (with a customized format containing MDC info) but also 1 log file to be used for Logstash/Elasticsearch with custom layout class. Would it also be possible to configure multiple file appenders using these properties?

[philwebb]

@marceloverdijk Sounds like a fairly complex scenario so it probably won't be possible with the properties. Have you tried defining your own logback.xml file? There are a couple imports that you can use (see this example).

[marceloverdijk]

Yes I'm using a custom logback.xml now. I only wished I could have a logback.xml per env.

Btw I think it should be possible to define multiple appenders via properties. With standard log4j properties file this is also possible. Thinking of it I wonder if it would be possible to define log4j.* properties (in applucation properties) and use them to initiate log4j? Just thinking loud to see options for customizations and per env config.

[joensson]

Have not tried it myself - but for environment specific logback configurations I think it is possible to do that using logback includes.

Create a 'master' logback configuration file that includes another environment specific logback-staging.xml - that has root tag 'included'.

logback-master.xml:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <include resource="logback-${LOGBACK_ENV}.xml"/>
    <include resource="org/springframework/boot/logging/logback/base.xml"/>
...
<!-- do something with the MY_PROPERTY value -->
...
</configuration>

logback-test.xml:

<?xml version="1.0" encoding="UTF-8"?>
<included>
    <property name="MY_PROPERTY" value="test"/>
...
</included>

When launching your application set the LOGBACK_ENV system property, e.g. System.setProperty("LOGBACK_ENV", "test") (this will of course not be necessary when the current issue has been fixed - at that time we can simple set a property in application.properties instead) and if the file exists you should see a log line from logback: INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback-test.xml] at ...

• almost lige we did in Spring in the good old pre-profile days when wanting different beans/properties per environment.

[philwebb]

I'm afraid I need to push this back for now :(

[marceloverdijk]

Hopefully another day maybe :-)

On 09 Dec 2014, at 19:31, Phil Webb notifications@github.com wrote:

I'm afraid I need to push this back for now :(

— Reply to this email directly or view it on GitHub.

[ljramones]

+1 agreed, this is a good feature.

[ghost]

+1 nice to have.

[brenuart]

+1 nice and avoids having to duplicate all of the standard logback.xml configuration files provided with SpringBoot.

[cmoro-deusto]

+1 thanks very much.

[txiong1987]

+1 Thanks!

[edudar]

+1 Can you go beyond just FILE_LOG_PATTERN here? I suspect this duplicates what @marceloverdijk asked above to some extend. We have custom Logback appender that talks to fluentd. Nothing fancy, new xml that include base one and has its own appender. There are 3 properties though: tag, host and port. We can configure them now through -D option but it would be nice to have streamlined way to define logging properties that will be replaced in the same way LOG_FILE with logging.file work.

[igler]

+1

[cannibalcow]

+1

[nagase2]

I also struggle with same issue because the log message issued twice. Eventually, I stop including base.xml at this moment... +1

[Chumper]

:+1: the same here, i want to use the AmqpAppender from spring-amqp

The problem is, that i need to get the host, username, password and port from the application config.

[iamiddy]

+1 need this feature, opened this SO before stumbled here

[hmeerlo]

+1

[cbelleza]

Missing it a lot... When will it be done?

[shaqtsui]

+1

[spencergibb]

+1 for spring-cloud-sleuth-correlation including a correlation id in the logs.

[ljramones]

What is spring cloud sleuth correlation? I am intrigued

[spencergibb]

@larry13767 see #2343 for the backstory. Some of the code has been donated from https://github.com/4finance/micro-infra-spring. It's an attempt to provide tracing from the original request throughout the system regardless of what machine sub-requests are made on.

[rob-baily]

I was looking at this question http://stackoverflow.com/questions/30886705/how-to-not-abbreviate-the-source-class-name-in-spriing-boots-loggger-name related to this request and it seems to me that the request to put them in the logging properties configuration may not be right. Spring Boot is supposed to be logging implementation agnostic and it looks like we want to duplicate what would be Logback configuration in the logging proprieties space. @philwebb Have you seen this code https://github.com/lukashinsch/spring-boot-extended-logging-properties? This seems like it might be a reasonable methodology to get some environment properties into the logging configuration and not have to replicate all logging configuration. Otherwise if you think that adding some Logback specific properties makes sense I could maybe give it a shot unless someone else is already looking at it and you have some idea on what the properties should be named.

[cbelleza]

any news about it?

[htynkn]

+1 any news?

[snicoll]

hey guys, this issue is scheduled for 1.3 (with a candidate on M2) so work on it should start soon. A PR would certainly speed things up!

[lukashinsch]

I gave it a shot last night (#3332), but I'm still struggling to get the CI build green (seems to be an environment dependent issue). In the meantime, I'd appreciate any feedback if the approach generally leads in the right direction.

[philwebb]

For anyone watching this issue there is now a new <springProperty> tag that you can use to pull in Environment properties. To use it you first need to rename your logback.xml file to logback-spring.xml. See the updated documentation for details. There's also a new <springProfile> tag that can be used to optionally include/exclude configuration.

I'd appreciate any feedback on this approach, does it work for people?

[ccit-spence]

How does this differ from using Spring Cloud Config? A couple of months ago I needed the functionality for determining which profile was loaded and was able to accomplish with Spring Cloud Config. Essentially loading profile specific logback configurations. Is this the same concept?

[philwebb]

@ccit-spence can you point me at a sample to show me what you mean?

[ccit-spence]

@philwebb I didn't have to really add anything except for configuring the service to use Spring Cloud Config It was really the only thing I have to do. In each service I have the following logback profile specific files. I do not have a logback.xml file. Only with its respective profile as below.

logback-local.xml
logback-docker.xml
logback-production.xml

Your example seems to add more robust capabilities. I am pretty sure the reason why Spring Cloud Config worked is that it refreshes the logback config after the initial load allowing for later processing during boot.

[philwebb]

@spencergibb @dsyer How does Spring Cloud Config support these profile specific logback configs? Is this something we should bring into Spring Boot? How does it deal with multiple active profiles?

[dsyer]

IIRC there is some hackery in Spring Cloud Commons to explicitly give log configuration a chance to refresh before starting to main application context. Remember that Spring Cloud always has a parent "bootstrap" context and it is there that you can set up things that need extra logic before the main context has started. I'm not sure there is anything magic about it in general, but logging systems require special treatment to get them initialized early, so I think that led to a special case.

[rob-baily]

@philwebb I like the approach as it seems very flexible. Based on this is it possible to update properties that are defined in defaults.xml like the CONSOLE_LOG_PATTERN and FILE_LOG_PATTERN from another property source? It seems like it would be useful to override just these and not completely define a new configuration just to override this.

[philwebb]

@rob-baily I've raised #3367 to consider this. We can't use the same <springProperty/> approach since base.xml might be imported from from a standard logback config where we don't have the extensions.

[rakesh9]

I am trying to use springProfile tag in logback.xml to have profile specific logging but spring boot is ignoring this and all tags after this.(I am using spring boot version 1.2.4 Release) -ERROR in ch.qos.logback.core.joran.spi.Interpreter@5:47 - no applicable action for [springProfile], current ElementPath is [[configuration][springProfile]]

-ERROR in ch.qos.logback.core.joran.spi.Interpreter@6:88 - no applicable action for [appender], current ElementPath is [[configuration][springProfile][appender]]

I tried putting the same configuration in logback-spring.xml but spring boot not recognizing and loading this file itself. Please guide me how to use springProfile tag to get profile specific logging.

[rob-baily]

@rakesh9 The new feature is coming is version 1.3.0. You can't use it until this unless you want to go against a SNAPSHOT build.

[philwebb]

@rakesh9 as @rob-baily said this is a new feature not and won't be available in 1.2.x. You'll also need to use logback-spring.xml as your filename when you do migrate (see the updated docs).

[rakesh9]

@rob-baily @ philwebb Thanks for the update..I missed observing the version...I am new to spring-boot.. can you suggest me any best alternative to have conditional logging for diff environments like dev,test using logback.groovy in 1.2.4 version

[tednology]

@philwebb Love the the new <springProfile> element in 1.3.0.M2, though I had a question:

The name attribute of <springProfile> doesn't seem to accept a list of profiles (I've tried with spaces, commas, semi-colons), is this working as intended?

[philwebb]

@edward-smith It only accepts a single name at the moment, but it should be easy to fix that. I'll open a new issue.

[thefalconfeat]

+1

[snicoll]

@thefalconfeat it's already implemented (and this issue is closed FWIW).

[lhanson]

Is there any support for this functionality in logback-spring.groovy, or is this XML-only?

[wilkinsona]

@lhanson It's XML only. We're not sure that it's possible with Logback's Groovy support. See #5051 for details.