Change the RMLStreamer logging level

RMLio / RMLStreamer

The RMLStreamer executes RML rules to generate high quality Linked Data from multiple originally (semi-)structured data sources in a streaming way.

http://rml.io/

MIT License

48 stars 18 forks source link

Change the RMLStreamer logging level #31

Open vemonet opened 3 years ago

vemonet commented 3 years ago

Hi @ghsnd , we would like to change the logging level to DEBUG in order to have more insights into the issues faced by the RMLStreamer (such as using functions)

The RMLStreamer is ran using the Flink CLI:

/opt/flink/bin/flink run -p 8 -c io.rml.framework.Main /mnt/RMLStreamer.jar toFile -m /mnt/mapping.rml.ttl -o /mnt/output.nt --job-name "RMLStreamer job"

We tried the following:

Changing the Apache Flink image config files: log4j.properties, log4j-cli.properties, log4j-console.properties to set the log4j.rootLogger property to DEBUG
Changing all log level to DEBUG in the RMLStreamer src/main/resources/log4j.properties file, then recompiling the RMLStreamer from the source code

But none of those attempts worked, what is the recommended way to change the logging level of the RMLStreamer?

vemonet commented 3 years ago

@bjdmeest @gertjandemulder any idea how the log level can be changed? It would be helpful for users of RMLStreamer to help you fix and improve it.

gertjandemulder commented 3 years ago

@vemonet , the logging can be configured using the log4j.logger.io.rml key in src/main/resources/log4j.properties

log4j.logger.io.rml=OFF
# no logs

log4j.logger.io.rml=FATAL
# no logs

log4j.logger.io.rml=ERROR
# logError messages are printed

log4j.logger.io.rml=WARN
# logError messages are printed
# logWarning messages are printed

log4j.logger.io.rml=INFO
# logError messages are printed
# logWarning messages are printed
# logInfo messages are printed

log4j.logger.io.rml=INFO
# logError messages are printed
# logWarning messages are printed
# logInfo messages are printed
# logDebug messages are printed

log4j.logger.io.rml=ALL
# logError messages are printed
# logWarning messages are printed
# logInfo messages are printed
# logDebug messages are printed

Hope this helps! Kind Regards, G

ghsnd commented 3 years ago

Hi @vemonet, If you want to change the log level of RMLStreamer running on a Flink cluster without changing the code, you can change <Flink root>/conf/log4j.properties. You can add two lines like this:

logger.rmlstreamer.name = io.rml.framework
logger.rmlstreamer.level = DEBUG

More info can be found in the Flink docs (link).

I'll add this info to the README.md

Kind regards, Gerald