Logging needs to be more flexible

[GoogleCodeExporter]

Currently our logging is all done by System.out.println (for results) and
System.err.println (for reports and warnings). These two levels of logging
are no longer sufficient for developer and user needs, now that some of the
components are used in several different code paths.

Should look into getting Java Logging to work, I've tried a bit without
much success but it's an issue to raise again in due course.

Original issue reported on code.google.com by dwidd...@gmail.com on 26 Mar 2009 at 11:19

[GoogleCodeExporter]

I think this is a rather important issue really: if this library is embedded 
into an application, the use of System.out and System.err can be irritating or 
just making the library unusable as is. This should be replaced by proper 
logging but even until then, it should be possible to suppress ALL output to 
System.out or System.err. 

The question is, which of the common logging frameworks (Log4j, Commons, slf4j, 
plain java) to use?

Original comment by johann.petrak@gmail.com on 23 Sep 2010 at 8:47

[GoogleCodeExporter]

I'd prefer plain java unless it's really really bad and one of the others is a 
ton better.

Would it be too egregious to start by just replacing all System.err.println 
with a single boilerplate logging statement?

Original comment by widd...@google.com on 23 Sep 2010 at 8:56

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

I guess it should be fine to replace all System.out.print/ln with logger.info 
and System.err.print/ln with logger.warning if processing continues or 
logger.severe if processing is terminated where logger would be the result of 
Logger.getLogger(CurrentClass.class.getCanonicalName()) or similar?

Original comment by johann.petrak@gmail.com on 23 Sep 2010 at 10:44

[GoogleCodeExporter]

Sounds right for all internal code. Some of the main binaries might still need 
to print to stdout since there are wrapper scripts that rely on this. 
Unfortunately we've been a bit loosey goosey in places with differentiating 
libraries from binaries, it might be time to improve this.

If you want to make a start, I'd be delighted to give you developer 
permissions! Otherwise it'll get done sometime, but probably not soon.

Thanks for your ongoing interest in the project, it is much appreciated.

Original comment by widd...@google.com on 24 Sep 2010 at 1:16

[GoogleCodeExporter]

Yes, I would start with those API classes which I am embedding into an 
application right now. Do you need any additional information from me for that 
(email is my name as shown at gmail.com)

Original comment by johann.petrak@gmail.com on 24 Sep 2010 at 8:07

[GoogleCodeExporter]

Just added you - feel free to check out the code, kick the tires, and let me 
know how you get on.

A word of warning - you'll find that a variety of different editors with 
different tabs and spaces settings have been used, for now feel free to 
reformat however is best for you, since we've been lax about style rules. My 
preference if we get round to implementing one would be no tabs and a regular 
2-space indent for nesting and 4-space indent for continued lines.

Original comment by widd...@google.com on 28 Sep 2010 at 1:30

[GoogleCodeExporter]

Sounds good, I use nearly the same conventions: no tabs, 2-space indent and 
continuation lines in a way that looks readable :)

Original comment by johann.petrak@gmail.com on 28 Sep 2010 at 3:06

[GoogleCodeExporter]

I am roughly planning on the following use of log levels:
SEVERE: something that is an obvious error and will prevent a useful result 
from being calculated; stack traces
WARNING: odd usage, potentially useless result or unwanted effects 
INFO: basic, high level information
FINE: verbose information
FINER: debugging information
FINEST: very verbose debugging information

Original comment by johann.petrak@gmail.com on 29 Sep 2010 at 9:02

[GoogleCodeExporter]

Sounds good to me.

I was once hoping that clients could modify logging levels, e.g., a main result 
for Search might be a small result along the way for ClusterResults. I tried 
and failed to get this to work. However, I think a proper factoring into binary 
and library classes will take care of this.

Original comment by widd...@google.com on 29 Sep 2010 at 9:05

[GoogleCodeExporter]

Its easily possible to modify the level and behavior of logging output by doing 
this:
* copy {JRE_HOME}/lib/logging.properties or create a new file with similar 
content
* set the properties according to taste, e.g. 
"java.util.logging.ConsoleHandler.level = FINEST"
* add the parameter 
-Djava.util.logging.config.file=path/to/mylogging.properties to the java 
command that calls your class

This will make the LoggingManager use those properties to set the logging.

Whether the level chosen by me for the various messages is apropriate is open 
to discussion/change ... I tried to make a judgement but your mileage might 
vary. I have no problem changing it to whatever others might prefer.

As far as binary classes are concerned (referring also to comment 5) I was a 
bit surprised to find that informatory messages are being written to System.err 
which seems to be against the convention of using System.err just for errors? 
Is this intended and does it need to be kept for backwards compatibilty?

Original comment by johann.petrak@gmail.com on 29 Sep 2010 at 9:56

[GoogleCodeExporter]

Nice that logging is configurable, I haven't tried this yet but thanks already. 
The levels you've chosen so far look fine.

No need to support writes to System.err, nobody should be depending on this. It 
was chosen just to make sure that clients who rely on System.out don't catch it 
by mistake, but there are surely better conventions.

Original comment by widd...@google.com on 30 Sep 2010 at 2:45

[GoogleCodeExporter]

Just adding my support for this change, I think its an important usability 
issue for the system. Certainly logging debug messages to Stderr is not ideal. 
Thanks for the efforts.

Original comment by bevan.ko...@gmail.com on 4 Nov 2010 at 1:44

[GoogleCodeExporter]

Some api classes actually output some kind of "progress" indicator to standard 
output (dots and the number of processed documents every N documents). I am not 
sure if it makes sense to write something similar to the log.
I think the correct way to implemt this would be to allow a client to register 
an progress event listener ... then it is up to the client to decide if and how 
to display the progress.
So the question is: should we keep the pregress indicator even if switching to 
logging, should we drop it entirely, or should we wait for an implementation 
based on events before dropping it?

Original comment by johann.petrak@gmail.com on 27 Nov 2010 at 5:57

[GoogleCodeExporter]

Most System.err.println statements have been replaced by Java logging 
statements, shipped in v 2.0, and no complaints as yet. The process counters 
are now a bit verbose, but this doesn't seem to have upset anyone yet.

There's still clean up to do, there are a few stray System.out.println in 
exceptions that should be logged instead, so I won't mark this as "fixed" yet. 
But we're most of the way there.

Original comment by widd...@google.com on 18 Jan 2011 at 3:14

• Changed state: Started