blendle / zapdriver

Blazing fast, Zap-based Stackdriver logging.
ISC License
176 stars 45 forks source link

Stacktrace error reporting #35

Open Dasio opened 4 years ago

Dasio commented 4 years ago

When I log error using

config := zapdriver.NewProductionConfig()
logger, err := config.Build(zapdriver.WrapCore(
    zapdriver.ReportAllErrors(true),
    zapdriver.ServiceName("service"),
))
logger.Error("something happened", zap.Error(errors.New("my error"))

I can see error in "Error reporting', no stacktrace. Stacktrace is in stacktrace field, but it is not recognized by GCP error reporting?

According to: https://cloud.google.com/error-reporting/reference/rest/v1beta1/projects.events/report#ReportedErrorEvent it should be included in message.

lyschoening commented 3 years ago

@JeanMertz Would you be willing to fix this or accept a PR for this issue?

bouk commented 3 years ago

My understanding is that the stack trace should be appended to the error message with two newlines between them. This might have to be done with a custom Encoder

hermanbanken commented 1 year ago

We have this issue too. Given a log like below the stacktrace is not reported in Error Reporting.

zap.Error("oops", zap.Error(detailedErr))

However, the logs contain this:

{
  "insertId": "aaaaaaaaaa",
  "jsonPayload": {
    "caller": "rpcs/some.go:161",
    "context": {
      "reportLocation": {
        "filePath": "/build/pkg/rpcs/some.go",
        "functionName": "github.com/redacted/redacted/pkg/rpcs.(*server).GetSomething",
        "lineNumber": "161"
      }
    },
    "error": "rpc error: code = Aborted desc = too much contention on these datastore entities. please try again. entity group key: Some/1L",
    "logging.googleapis.com/traceSampled": false,
    "message": "oops",
    "stacktrace": `github.com/redacted/redacted/pkg/rpcs.(*server).GetSomething
    /build/pkg/rpcs/some.go:161
  github.com/redacted/shared-proto/auth/gen/go._Server_GetSomething_Handler.func1
    /build/pkg/shared-proto/auth/gen/go/something_grpc.pb.go:118
  github.com/redacted/redacted/pkg/kit.instrumentCalls
    /build/pkg/kit/server.go:180
  github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
    /go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/chain.go:25
  go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc.UnaryServerInterceptor.func1
    /go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc@v0.16.0/interceptor.go:333
  github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
    /go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/chain.go:25
  github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1
    /go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.2.2/chain.go:34
  github.com/redacted/shared-proto/auth/gen/go._Server_GetSomething_Handler
    /build/pkg/shared-proto/auth/gen/go/something_grpc.pb.go:120
  google.golang.org/grpc.(*Server).processUnaryRPC
    /go/pkg/mod/google.golang.org/grpc@v1.50.0/server.go:1318
  google.golang.org/grpc.(*Server).handleStream
    /go/pkg/mod/google.golang.org/grpc@v1.50.0/server.go:1659
  google.golang.org/grpc.(*Server).serveStreams.func1.2
    /go/pkg/mod/google.golang.org/grpc@v1.50.0/server.go:955`,
  "timestamp": "2023-01-26T12:13:21.585164121Z"     
  }
}

Therefore, simply moving stacktrace to stack_trace does not solve the issue. StackDriver will only use stack_trace and ignore message:

docs: You can specify more than one of those fields. If more than one of those fields is specified, then the order of evaluation is: stack_trace, then exception, and then message.

So maybe we should use a custom encoder and wrap the error inside the zap.Error with message and put that in message instead.