Consistent logging/error interface/debug verbosity levels for API / CLI

[cboulanger]

There should be a consistent way of dealing with

• [ ] user feedback / debugging messages (API vs CLI)
• [ ] error handling (API vs CLI)
• [ ] verbosity of output (API vs CLI)
• [ ] access to these settings in all commands through options in the CLI
• [ ] output channel (stdout/stderr) for CLI
• [ ] exit codes for CLI

[cboulanger]

So maybe instead of having --verbose vs --quiet, we could have --verbosity X, x being

0 - no output, total silence
1 - only errors
2 - info & errors & warnings
3 - increased verbosity, to get more information on what's going on
4 - plus debug output & stack traces (development only)
5 - plus heavy duty debug, including for example object dumps  (development only)

I don't want to reinvent the wheel, though, so we might just adopt an existing pattern. We could still add option names as syntactic sugar on top of the numeric debug levels. @johnspackman , @derrell, what do you think?

[johnspackman]

—quiet and —q is a very natural and common usage for reducing output, and ultimately we want qooxdoo to be installed via the npm in which case qxpath will normally be ./node_modules/qooxdoo-sdk. This would suggest that —qxpath is less likely to be used than —quiet, so -q would be better aliased to —quiet. The —qxpath could be aliased to —qx … or maybe not aliased at all

IMHO there is a distinction between code debug output (i.e. where this.debug / this.error / etc go) and compiler status (ie compiler warnings and errors). The this.debug is a plain untranslated English string, that is added and removed from time to time, and completely focused on the development side of things. This is already controllable via qx.log.* by adding filters (or will be when the qooxdoo-cli/compiler upgrade to Qooxdoo 6.0/master), and that gives ideal flexibility when tracking down problems in code that you cannot attach a debugger to.

The "compiler status" output includes compiler errors, warnings, and some status information - these need to be translatable strings; in our code they would be represented by some unique identifier like "symbol.unresolved" or "defer.unsafe", and we could associate a verbosity level or bitmask with each one so that we can easily adjust what get displayed according to a "verbosity" level.

As you can probably guess from those two examples, I've already made a start on this in the compiler, in that it associates "markers" with points in a source code file, the marker being a warning or error.

One of my goals with the "markers" was to have a verbosity level where the message ID and it's arguments are output, ie without any human readable, translated strings because this would allow for machine-readable output from the compiler. So for example, a process inside the web server can choose to reload the server application when it gets recompiled by qx compile --watch

The verbosity levels above cross over both of these worlds, but as this.debug is covered by qx.log.* filters, we can reduce the verbosity level down to only this:

0 - no output, total silence
1 - only errors
2 - info & errors & warnings
99 - error *codes* only

[cboulanger]

@johnspackman Thanks for clearing things up, makes complete sense to me! I guess this provides even more good reasons to use the qooxdoo framework on the server.

[oetiker]

when designing command line interfaces I usually try to get inspiration from existing well known tools ...

samba for example lets you specify the subsystem you want to set an error level for

[cboulanger]

@johnspackman Are your "markers" numeric codes that can be used as the exit code for the CLI commands?

[johnspackman]

No, because there are potentially so many of them - those codes really refer to one line of output.

Perhaps we could say that the error code relates to the amount of warnings/errors which is output? EG 0 for no errors/warnings, 1 for some warnings, 2 for errors and warnings. But if you set --verbosity=1, then you will only ever get 0 or 2.

[cboulanger]

Hm, @johnspackman can you elaborate? I don't fully understand: I thought verbosity would refer to debug level (meant for humans), whereas exit codes are for consumption by scripts...

@derrell, you brought up the question of exit codes, maybe you have a take on this?

[cboulanger]

As @oetiker suggested, maybe we an adopt an exisiting pattern compatible with @johnspackman's ideas - we aren't the first to solve these problems, I assume 😄

[johnspackman]

For example "symbol.unresolved" - this marker will be reported in (potentially) lots of places where there is an unresolved symbol, so I just meant that it's not a 1-to-1 for choosing an exit code.

It depends what the exit code is used for, but ultimately I'm just suggesting that the exit code would be used to report whether one or more errors were reported during compilation. Whether it would also be useful to report warnings may depend on the use case, but I'm easy either way.

Slightly more sophisticated error detection could be implemented with a simple bash script and grep/awk/etc, or as everything is now an API delivered via npm packaging there is a very structured and easy way for really advanced users to make it work exactly the way they want.

We don't have many use cases other than the old generator - I've just grep-ed the code and AFAICT it was exit code 1 if Qooxdoo could not be found or the config.json could not be parsed, otherwise it was 0.

[cboulanger]

@johnspackman thanks for the explanation: I am also in favour of making things as little complex as possible while allowing for extended functionality to be implemented later if needed. So I would be totally fine with exit codes 0 and 1, and scripts accessing log files if they really need to know what the problem was. So to sum up, we would have:

• message markers which can be translated (to be consistently implemented everywhere)
• verbosity levels regulating what goes into the the log appender (which outputs either to a file or to stdout/stderr (to be implemented)
• log levels consistent with the qooxdoo approach (debug, info, warn, error) used by the qooxdoo-server based command scripts (already implemented in qooxdoo code)
• exit codes 0 for success and 1 for errors (to be implemented)

Did I get you correctly?

[johnspackman]

Yes, except that I would make the distinction between "logging" (as in Qooxdoo debug/info/warn/error) and other, more formal output (let's call it "Console"??). So the "verbosity levels" would relate to the Console output, and Qooxdoo log levels and filtering relate to debug/info/warn/error.

And in production, by default the Logging is never shown or silently goes to a log file.

[johnspackman]

This issue was moved to qooxdoo/qooxdoo-compiler#78