apache / rocketmq-clients

RocketMQ Clients - Collection of Client Bindings for Apache RocketMQ
https://rocketmq.apache.org/
Apache License 2.0
305 stars 202 forks source link

[Feature] 接入 OpenTelemetry 链路追踪,方便消息跟踪 #703

Open sduo opened 6 months ago

sduo commented 6 months ago

Programming Language of the Client

C#

Is Your Feature Request Related to a Problem?

C# 客户端无法对消息进行追踪

Describe the Solution You'd Like

通过向 DiagnosticListener 写入相关事件,为消息加入相关的追踪数据。

Describe Alternatives You've Considered

=== SDK 修改开始 ===

  1. 定义一个名称为 Org.Apache.Rocketmq 的 DiagnosticListener。
  2. 在 ClientManager 发送消息时,向 DiagnosticSource 中写入 SendMessage 事件,载荷为当次发送的所有消息的数组。
  3. 在 ClientManager 接收消息时,向 DiagnosticSource 中写入 ReceiveMessage 事件,载荷为当次接收的所有消息的数组。 === SDK 修改完毕 ===

=== OpenTelemetry 实现开始 ===

  1. 此时可在外部对通过 DiagnosticListener 订阅名称为 Org.Apache.Rocketmq 的事件。
  2. 在 SendMessage 事件中,依次遍历载荷中的消息,判断消息的 SystemProperties.TraceContext 是否包含相关数据。 5.1 包含跟踪数据,不做任何处理,放行即可。 5.2 不包含跟踪数据,写入跟踪信息,并上报。
  3. 在 ReceiveMessage 事件中,依次遍历载荷中的消息,判断消息的 SystemProperties.TraceContext 是否包含相关数据。 6.1 不包含跟踪数据,不做任何处理,放行即可。 6.2 包含跟踪数据,读取跟踪信息,并上报。 === OpenTelemetry 实现完毕 ===

Additional Context

No response

aaron-ai commented 6 months ago

Nice proposal.

sduo commented 6 months ago

@aaron-ai 经过进一步的思考与论证,打算将SendMessageReceiveMessage拆分成4个,对应事件及数据来源打算按照下表:

其中:?处有待商讨!

Attribute SendMessageBefore SendMessageAfter ReceiveMessageBefore ReceiveMessageAfter
Span | Id | Activity.Id | - | - | Activity.Id
Span | ParentId | - | Message.UserProperties.TraceContext ? | - | Message.SystemProperties.TraceContext
Tag | messaging.system | - | "rocketmq" | - | "rocketmq"
Tag | messaging.client_id | - | Metadata["x-mq-client-id"] | - | Metadata["x-mq-client-id"]
Tag | messaging.operation | - | "publish" | - | "receive" ? "deliver" ?
Tag | messaging.destination.name | - | Message.Topic | - | Message.Topic
Tag | messaging.destination.anonymous - - - -
Tag | messaging.destination.template - - - -
Tag | messaging.destination.temporary - - - -
Tag | messaging.destination_publish.name - - - -
Tag | messaging.destination_publish.anonymous - - - -
Tag | messaging.message.conversation_id | - | Metadata["x-mq-request-id"] ? | - | Metadata["x-mq-request-id"] ?
Tag | messaging.message.id | - | Message.SystemProperties.MessageId | - | Message.SystemProperties.MessageId
Tag | messaging.rocketmq.message.type | - | Message.SystemProperties.MessageType | - | Message.SystemProperties.MessageType
Tag | messaging.rocketmq.message.tag | - | Message.SystemProperties.Tag | - | Message.SystemProperties.Tag
Tag | messaging.rocketmq.namespace | - | ClientConfig.Endpoints ? | - | ClientConfig.Endpoints ?
Tag | messaging.rocketmq.message.keys | - | Message.SystemProperties.Keys | - | Message.SystemProperties.Keys
Tag | messaging.rocketmq.message.group | - | Message.SystemProperties.MessageGroup | - | Message.SystemProperties.MessageGroup
Tag | messaging.message.envelope.size - - - -
Tag | messaging.message.body.size | - | Message.Body.Length | - | Message.Body.Length
Tag | messaging.rocketmq.client_group | - | - | - | ReceiveMessageRequest.Group.Name
Tag | messaging.batch.message_count | - | - | - | ReceiveMessageRequest.BatchSize
Tag | messaging.rocketmq.consumption_model - - - -
Tag | messaging.rocketmq.message.delay_time_level - - - -
Tag | messaging.rocketmq.message.delivery_timestamp | - | Message.SystemProperties.DeliveryTimestamp | - | Message.SystemProperties.DeliveryTimestamp
Tag | messaging.gcp_pubsub.message.ordering_key - - - -

论证点:

  1. 不能直接暴露ClientConfig,其中包含了认证敏感信息。
  2. 另外Metadata信息中,我觉得可以有更多的关键数据被记录下来,但无对应的Tag属性。
项目 数值
x-mq-language | "DOTNET"
x-mq-client-version | "5.1.1-rc1"
x-mq-request-id | GUID
github-actions[bot] commented 5 months ago

This issue is stale because it has been open for 30 days with no activity. It will be closed in 3 days if no further activity occurs.

ZUOXIANGE commented 1 week ago

这个Feature不错,现在有计划实现吗