Output not compatible with the GKE integration

ricmalta commented 6 years ago

I'm doing a basic integration with negroni and logrus middleware using this formatter . However the format is not being recognised by stackdriver. My Implementation:

utils.logger.go

package utils

import (
    "os"

    stackdriver "github.com/TV4/logrus-stackdriver-formatter"
    "github.com/sirupsen/logrus"
)

// NewLogger instance
func NewLogger(name string) (*logrus.Logger, error) {
    logger := logrus.New()
    logger.Out = os.Stdout
    logger.Formatter = stackdriver.NewFormatter(
        stackdriver.WithService(name),
        stackdriver.WithVersion("v0.1.0"),
    )

    return logger, nil
}

Middleware implementation

// NewMainMiddleware creates a middleware with common app middlewares
func NewMainMiddleware(middleware *negroni.Negroni, apiConfig config.APIConfig) *negroni.Negroni {
    log, _ := utils.NewLogger("http")

    return middleware.
        With(negroni.NewRecovery()).
        With(negronilogrus.NewMiddlewareFromLogger(log, "http")).
        With(LanguageMiddleware{}).
        With(timeoutMiddleware(apiConfig))
}

Output sample

{"timestamp":"2018-05-15T12:50:47Z","message":"completed handling request","severity":"INFO","context":{"data":{"measure#http.latency":193507,"method":"GET","remote":"10.132.0.2:60088","request":"/v1/healthz","status":200,"text_status":"OK","took":193507}}}
{"timestamp":"2018-05-15T12:50:51Z","message":"started handling request","severity":"INFO","context":{"data":{"method":"GET","remote":"10.132.0.2:60140","request":"/v1/healthz"}}}
{"timestamp":"2018-05-15T12:50:51Z","message":"completed handling request","severity":"INFO","context":{"data":{"measure#http.latency":70072451,"method":"GET","remote":"10.132.0.2:60140","request":"/v1/healthz","status":200,"text_status":"OK","took":70072451}}}```

Stackdriver output

{
 insertId:  "1vpqxs3g2enyp0w"  
 jsonPayload: {
  context: {
   data: {
    measure#http.latency:  164545     
    method:  "GET"     
    remote:  "*******:*****"     
    request:  "/v1/healthz"     
    status:  200     
    text_status:  "OK"     
    took:  164545     
   }
  }
  message:  "completed handling request"   
  timestamp:  "2018-05-15T12:45:41Z"   
 }
 labels: {
  compute.googleapis.com/resource_name:  "fluentd-gcp-v2.0.9-zv825"   
  container.googleapis.com/namespace_name:  "*******-staging"   
  container.googleapis.com/pod_name:  "api-*******-service-api-********-*****"   
  container.googleapis.com/stream:  "stdout"   
 }
 logName:  "projects/********-*****/logs/*******-service-api"  
 receiveTimestamp:  "2018-05-15T12:45:44.697156895Z"  
 resource: {
  labels: {
   cluster_name:  "*******-gke"    
   container_name:  "*******-service-api"    
   instance_id:  "*****************"    
   namespace_id:  "*******-staging"    
   pod_id:  "*******-service-api-********-*****"    
   project_id:  "**********"    
   zone:  "*********"    
  }
  type:  "container"   
 }
 severity:  "INFO"  
 timestamp:  "2018-05-15T12:45:41Z"  
}

As you can see the playload is in the jsonPayload.context.data and not being actually filtered by stackdriver. By your code, the fields are not completely compatible with the current (v2) LogEntry definition: https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry.

There is any easy solution to this or this module is deprecated?