cap-js / telemetry

CDS plugin providing observability features, incl. automatic OpenTelemetry instrumentation.
https://cap.cloud.sap/docs
Apache License 2.0
8 stars 7 forks source link

Option to export/sample only error traces (support own sampler?) #183

Closed jravnik closed 3 months ago

jravnik commented 5 months ago

I would like to export traces/spans only if they contain an error (effectively filtering out all non-error traces). This would allow to focus on error monitoring and drastically reduce the amount of trace data exported and processed, especially since all cloud storage/runtime providers charge based on resource usage.

I think it would require its own sampler. But that doesn't work with the current implementation, right? https://github.com/cap-js/telemetry/blob/c04363281416effff0569199ce853ff405d6c74c/lib/tracing/index.js#L36-L45

How can this be realized? Is there maybe already a configuration that I am not aware of?

Thanks!

sjvans commented 3 months ago

hi @jravnik

the sampler decides, which incoming requests to trace. at that time, you don't yet know whether it will resolve in an error or not.

instead, you'd need to provide a custom exporter, which you can do like this:

"exporter": {
  "module": "./lib/MyOTLPTraceExporter", //> path to js file
  "class": "MyOTLPTraceExporter" //> name of exported class
}

(cf. https://github.com/cap-js/telemetry/blob/main/README.md#exporters)

with something like so:

class MyOTLPTraceExporter extends OTLPTraceExporter {
    export(spans, resultCallback) {
        if (DONT_EXPORT) return resultCallback({ code: 0 })
        super.export(spans, resultCallback)
    }
}

best, sebastian