search

openhab / openhab-distro

The binary distribution of openHAB
https://www.openhab.org/
Eclipse Public License 2.0
1.3k stars 391 forks source link

Split log configuration into system and user entries #516

Open kaikreuzer opened 7 years ago

kaikreuzer commented 7 years ago

Currently, the logging configuration is stored in userdata/etc/org.ops4j.pax.logging.cfg. This file either gets overwritten by updates or not - but we do not have a chance to do adjustments in the system configuration without wiping any changes the user might have done.

As we have moved to log4j2 now, there is chance to address this: log4j2 supports multiple configuration files. It would therefore be ideal to have the system config stored in runtime/etc, while the user config could reside in conf or userdata/etc (in case it is possible to edit the file e.g. through logging changes in the console).

cweitkamp commented 7 years ago

@kaikreuzer Can you give any hint where to start looking to implement this feature?

kaikreuzer commented 7 years ago

Not really. I guess 80% of the work for this issue is finding the right place and way how to implement it :-) The best starting point for the investigation would imho be the new feature in log4j2 itself and the Karaf logging documentation - this content about log4j2 wasn't there yet last week, so it must be pretty fresh :-)

We might think about moving from property files to XML files for the configuration as well (as it is also used in the docs now) - I somewhere read that XML includes should also work, so this might be another option for split files.

One thing to also keep in mind is how to deal with logging configuration changes through the Karaf shell - the log:set commands update the logging configuration files, which can be quite neat - but we must also ensure that nothing is written to conf and runtime as these folders are considered to be read-only.

Hope this helps to get an idea about the issue and where to look at!

cweitkamp commented 7 years ago

I did some research. At the first try I got totally lost and gave up. A few days ago I coincidentally saw your comment in openhab/openhab-docs#476 containing the link to the new released documentation of Karaf logging. This manual is much better than the old version. Here are some facts I discovered so far. Now we have to put them together:

The first three points definitely makes sense and I am sure we can work out a solution for our problem statement. The fourth point sounds very interesting, but doesn't really fit into the scheme. It seems to be an option, if you are able to set it in some lines of code (see example).

wborn commented 6 years ago

The fourth point sounds very interesting, but doesn't really fit into the scheme. It seems to be an option, if you are able to set it in some lines of code (see example).

I don't think you need to add it to a piece of code. You can just set it using the JAVA_OPTS environment variable. I've used it before to suppress logging in Maven builds. :-)

E.g. -Dlog4j.configurationFile=log4j2-A.xml,log4j2-B.xml,log4j2-C.xml

kaikreuzer commented 6 years ago

@wborn So would it be as simple as adding such a parameter to our setenv scripts?

wborn commented 6 years ago

After a very very brief look I don't think Pax will pickup such logging configurations so it will probably not be visible/editable from the console. You'll be directly configuring Log4j2 with this and have to hope other logic leaves such configuration alone.

wborn commented 6 years ago

You can add a log4j v2 XML-based configuration file by adding this line to the above file org.ops4j.pax.logging.log4j2.config.file=${karaf.etc}/log4j2.xml

There is an issue about log:get and log:set commands not properly working when using a log4j2.xml file (see KARAF-5354). So for proper functionality we would need to upgrade to Karaf 4.2.0 or newer.

kaikreuzer commented 6 years ago

Hm, ok, so nothing for the 2.2 release then. Ok for me to live with the single file for the moment.

wborn commented 6 years ago

I've made local build with OH 2.2.0 and Karaf 4.2.0.M1 and tested the implementation of using a log4j2.xml:

So in its current state you can not easily use it for splitting up configurations with XIncludes. It would be a nice replacement for properties files though!

AngelosF commented 5 years ago

Is this still worth pursuing ? (or what @wborn mentioned in the last comment are show stoppers?)

I have an idea about a proper implementation but I have some questions:

1) Where should we place the log config file? 2) Which format should we use? (I like xml the best versus json and/or yaml)

etc/org.ops4j.pax.logging.cfg only 1 line:

org.ops4j.pax.logging.log4j2.config.file=${openhab.etc}/services/log4j2.???

I can make some tests and propose a solution.

hakan42 commented 5 years ago

Is this still worth pursuing ? (or what @wborn mentioned in the last comment are show stoppers?) I can make some tests and propose a solution.

I'd love whatever solution allows me to split up the log files according to bindings. This would help tremendously, at least while developing.

kaikreuzer commented 5 years ago

@AngelosF Yes, definitely worth pursuing!

Where should we place the log config file?

conf/ or conf/services/ for the user editable parts runtime/etc for the static (read-only) system parts

Which format should we use? (I like xml the best versus json and/or yaml)

It seems as if everyone is using XML for log4j2, so we should do the same.

splatch commented 5 years ago

FYI it is possible to have "per bundle" logging thanks to MDC/NDC - see configuration sample https://issues.apache.org/jira/browse/LOG4J2-2021.

rkoshak commented 5 years ago

I can make some tests and propose a solution.

@AngelosF, did you make any progress on this in your tests? I started looking into this too but I don't want to waste my time if you've already discovered some limitations or problems.