Support different Stackdriver log levels

[oliver-goetz]

As of today many cloud robotics apps are writing all their console logs to stderr. Thus, they are listed in the error category of Stackdriver logging and it is hard to scan logs for real errors.

There are some ways to fix this:

• writing info messages to stdout and just error messages to stderr
• implement JSON logging and set the severity property according to the log level

This issue affects following apps:

• cloud-master
• robot-master
• cr-syncer
• token-vendor
• metadata-server
• gcr-credential-refresher

[ensonic]

For the components written in go, we could try if changing log.Print() output to fmt.Print() (which goes to stdout) to see if that make the output show up as INFO. Unfortunately the format does not contain markers for stackdriver to detect the severity.

The token vendor is written in java and here the logs clearly have markers:

2020-05-18 12:51:35 INFO com.cloudrobotics.tokenvendor.CloudIotTokenVerifier verifyToken checking ....

Need to follow-up with the stackdriver team to see why it is not recognized. Some more info here: https://cloud.google.com/logging/docs/setup/java

[oliver-goetz]

Even without JSON logging it could make sense to create different log handler in Go to format logs according to the log level. Something like this but with configurable io.Writers.

var (
    logTrace   *log.Logger
    logInfo    *log.Logger
    logWarning *log.Logger
    logError   *log.Logger
)

func initLog(
    traceHandle io.Writer,
    infoHandle io.Writer,
    warningHandle io.Writer,
    errorHandle io.Writer) {

    logTrace = log.New(traceHandle,
        "TRACE: ",
        log.Ldate|log.Ltime|log.Lshortfile)

    logInfo = log.New(infoHandle,
        "INFO: ",
        log.Ldate|log.Ltime|log.Lshortfile)

    logWarning = log.New(warningHandle,
        "WARNING: ",
        log.Ldate|log.Ltime|log.Lshortfile)

    logError = log.New(errorHandle,
        "ERROR: ",
        log.Ldate|log.Ltime|log.Lshortfile)
}

func init() {
        initLog(ioutil.Discard, os.Stdout, os.Stdout, os.Stderr)
}

In Java the logger instances are created using FluentLogger.forEnclosingClass(). According to its documentation it uses "the default parser and system configured backend". This seems to be configured to java.util.logging.ConsoleHandler which logs to System.err