Open yijiem opened 1 month ago
Good point, do we need a new class to describe any
and a new variant
which contains map<string, any>
? It may changes the ABI. @open-telemetry/cpp-approvers
To be specs compliant with Log data model, we would need to add support for Any
type with below variant types in addition to those existing in AttributeValue :
Supporting non-owning types for these heterogenous deeply nested structure would be complex to implement and maintain, and I believe we can wait for sufficient ask for this support before deciding on it.
As of now, other languages with stable log API - Java, JS, .Net, Python - are not fully compliant as well. Java uses string type for body, python reuses common attributes for log-attributes. I think most of the languages are taking decision based on whether the logging frameworks for these language support these types or not. And in C++, we provide a user-facing logging API where it is recommended to use the message-template format for the log body.
Will the message-template
formatted log body and its structure being preserved when OTel-Cpp exports the log data e.g. through OTLP to a collector (and eventually to a backend logging store)? gRPC wants to log JSON-structured data through OTel and ideally the structure of the data should be preserved in the backend logging store.
BTW, just throw in one idea of implementing a JSON-like nested data structure: https://godbolt.org/z/oeEh66aad.
Will the
message-template
formatted log body and its structure being preserved when OTel-Cpp exports the log data e.g. through OTLP to a collector (and eventually to a backend logging store)? gRPC wants to log JSON-structured data through OTel and ideally the structure of the data should be preserved in the backend logging store.BTW, just throw in one idea of implementing a JSON-like nested data structure: godbolt.org/z/oeEh66aad.
OTel-Cpp is not supposed to change or format the log body, the message-template
and attributes will be kept and exported.
For logging JSON, wondering that is the benefit of structured with map<string, any>
than logging it as a plain string?
Structured logging in supported backend logging stores will have benefits such as easy to query. This is especially true when the payload is structured at the point of its origin.
Edit: it seems like OTLP is able to carry structured data in LogRecord.body.
discussed in SIG meeting 7th August, https://github.com/open-telemetry/opentelemetry-cpp/issues/3000#issuecomment-2222150596 and https://github.com/open-telemetry/opentelemetry-cpp/issues/3000#issuecomment-2224043461 we would support only string type as body.
How about supporting structured data type as body in the Event API which IIUC is for logging structured data with well-defined semantic/schema and requires an AnyValue
typed Body
field in its spec as well? Either way there needs to be an AnyValue
type in opentelemetry-cpp.
IIUC, opentelemetry-java is in the process of stablizing AnyValue and APIs for recording AnyValue log record bodies, see https://github.com/open-telemetry/opentelemetry-java/issues/6581, https://github.com/open-telemetry/opentelemetry-java/pull/6591 and https://github.com/open-telemetry/opentelemetry-java/issues/6626 which will make them compliant with the Log data model.
opentelemetry-go already supports this since the language itself supports any type: https://github.com/open-telemetry/opentelemetry-go/blob/main/log/keyvalue.go#L48-L52.
In the OTel Logs Data Model, it is specified that the Body field must support
any
type to preserve the semantics of structured logs emitted by the application: https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-body. This is specifically useful for the application which logs a JSON-structured data i.e.map<string, any>
as the Body. But the OTel C++ Logger API does not support such a case, it supportsstring_view
or AttributeValue type in the Body (code) but neither of those can preserve the semantics of structured data.Ideally, if the Logger API preserves the semantics of structured data, the structure should be preserved across the stack (e.g. from application to OTel collector via OTLP and eventually the structured data gets exported to the backend log store).
Seems like there is also an Event API for logging structured data. But it is still a WIP and is a specialized Log API and would have the same issue in opentelemetry-cpp since the current API does not support structured data.