demo - Githubissues

zking2000 / NotePad

1 stars 0 forks source link

demo #74

Open zking2000 opened 4 days ago

zking2000 commented 4 days ago

apiVersion: apps/v1
kind: Deployment
metadata:
  name: flask-api-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: flask-api
  template:
    metadata:
      labels:
        app: flask-api
    spec:
      containers:
      - name: flask-api
        image: api:v2
        ports:
        - containerPort: 5000
        env:
        - name: OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED
          value: "true"
        - name: OTEL_SERVICE_NAME
          value: "flask-api"
        - name: OTEL_RESOURCE_ATTRIBUTES
          value: "service.namespace=grafana-stack"
        - name: OTEL_LOG_LEVEL
          value: "DEBUG"
        - name: OTEL_TRACES_SAMPLER
          value: "always_on"
        - name: OTEL_TRACES_EXPORTER
          value: "otlp"
        - name: OTEL_TRACES_EXPORTER_OTLP_ENDPOINT
          value: "localhost:4317"
        - name: OTEL_TRACES_EXPORTER_OTLP_INSECURE
          value: "true"
        - name: OTEL_LOGS_EXPORTER
          value: "otlp"
        - name: OTEL_LOGS_EXPORTER_OTLP_ENDPOINT
          value: "http://localhost:4318"
        - name: OTEL_LOGS_EXPORTER_OTLP_INSECURE
          value: "true"
      - name: otel-collector
        image: otel/opentelemetry-collector:0.109.0
        ports:
        - containerPort: 4317
        - containerPort: 4318
        volumeMounts:
        - name: otel-collector-config
          mountPath: /etc/otelcol/config.yaml
          subPath: config.yaml
      volumes:
      - name: otel-collector-config
        configMap:
          name: otel-collector-config
---
apiVersion: v1
kind: Service
metadata:
  name: flask-api-service
spec:
  selector:
    app: flask-api
  ports:
    - protocol: TCP
      port: 80
      targetPort: 5000
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: user-api-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: user-api
  template:
    metadata:
      labels:
        app: user-api
    spec:
      containers:
      - name: user-api
        image: user:v2
        ports:
        - containerPort: 5000
        env:
        - name: OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED
          value: "true"
        - name: OTEL_SERVICE_NAME
          value: "user-api"
        - name: OTEL_RESOURCE_ATTRIBUTES
          value: "service.namespace=grafana-stack"
        - name: OTEL_LOG_LEVEL
          value: "DEBUG"
        - name: OTEL_TRACES_SAMPLER
          value: "always_on"
        - name: OTEL_TRACES_EXPORTER
          value: "otlp"
        - name: OTEL_TRACES_EXPORTER_OTLP_ENDPOINT
          value: "localhost:4317"
        - name: OTEL_TRACES_EXPORTER_OTLP_PROTOCOL
          value: "grpc"
        - name: OTEL_TRACES_EXPORTER_OTLP_INSECURE
          value: "true"
        - name: OTEL_LOGS_EXPORTER
          value: "otlp"
        - name: OTEL_LOGS_EXPORTER_OTLP_ENDPOINT
          value: "http://localhost:4318"
        - name: OTEL_LOGS_EXPORTER_OTLP_INSECURE
          value: "true"
      - name: otel-collector
        image: otel/opentelemetry-collector:0.109.0
        ports:
        - containerPort: 4317
        - containerPort: 4318
        volumeMounts:
        - name: otel-collector-config
          mountPath: /etc/otelcol/config.yaml
          subPath: config.yaml
      volumes:
      - name: otel-collector-config
        configMap:
          name: otel-collector-config
---
apiVersion: v1
kind: Service
metadata:
  name: user-api-service
spec:
  selector:
    app: user-api
  ports:
    - protocol: TCP
      port: 80
      targetPort: 5000
  type: ClusterIP
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: otel-collector-config
data:
  config.yaml: |
    receivers:
      otlp:
        protocols:
          grpc:
          http:

    processors:
      batch: {}
      resource:
        attributes:
        - action: upsert
          key: service.name
          value: "flask-api"
        - action: upsert
          key: service.namespace
          value: "grafana-stack"

    exporters:
      debug:
        verbosity: detailed
      otlp/tempo:
        endpoint: "opentelemetry-collector.grafana-stack.svc.cluster.local:4317"
        tls:
          insecure: true
      otlphttp/loki:
        endpoint: "http://opentelemetry-collector.grafana-stack.svc.cluster.local:4318"
        tls:
          insecure: true

    service:
      pipelines:
        traces:
          receivers: [otlp]
          processors: [batch, resource]
          exporters: [otlp/tempo, debug]
        logs:
          receivers: [otlp]
          processors: [batch, resource]
          exporters: [otlphttp/loki, debug]

zking2000 commented 4 days ago

# OTEL-COLLECTOR

---
# Source: opentelemetry-collector/templates/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: opentelemetry-collector
  namespace: grafana-stack
  labels:
    helm.sh/chart: opentelemetry-collector-0.106.0
    app.kubernetes.io/name: opentelemetry-collector
    app.kubernetes.io/instance: opentelemetry-collector
    app.kubernetes.io/version: "0.109.0"
    app.kubernetes.io/managed-by: Helm
---
# Source: opentelemetry-collector/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: opentelemetry-collector
  namespace: grafana-stack
  labels:
    helm.sh/chart: opentelemetry-collector-0.106.0
    app.kubernetes.io/name: opentelemetry-collector
    app.kubernetes.io/instance: opentelemetry-collector
    app.kubernetes.io/version: "0.109.0"
    app.kubernetes.io/managed-by: Helm

data:
  relay: |
    exporters:
      debug:
        verbosity: detailed
      otlp/tempo:
        endpoint: tempo-distributor:4317
        tls:
          insecure: true
      otlphttp/loki:
        endpoint: http://loki:3100/loki/api/v1/push
    extensions:
      health_check:
        endpoint: ${env:MY_POD_IP}:13133
    processors:
      batch: {}
    receivers:
      otlp:
        protocols:
          grpc:
            endpoint: 0.0.0.0:4317
          http:
            endpoint: 0.0.0.0:4318
    service:
      extensions:
      - health_check
      pipelines:
        logs:
          exporters:
          - debug
          - otlphttp/loki
          processors:
          - batch
          receivers:
          - otlp
        traces:
          exporters:
          - debug
          - otlp/tempo
          processors:
          - batch
          receivers:
          - otlp
---
# Source: opentelemetry-collector/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: opentelemetry-collector
  namespace: grafana-stack
  labels:
    helm.sh/chart: opentelemetry-collector-0.106.0
    app.kubernetes.io/name: opentelemetry-collector
    app.kubernetes.io/instance: opentelemetry-collector
    app.kubernetes.io/version: "0.109.0"
    app.kubernetes.io/managed-by: Helm

    component: standalone-collector
spec:
  type: ClusterIP
  ports:

    - name: jaeger-compact
      port: 6831
      targetPort: 6831
      protocol: UDP
    - name: jaeger-grpc
      port: 14250
      targetPort: 14250
      protocol: TCP
    - name: jaeger-thrift
      port: 14268
      targetPort: 14268
      protocol: TCP
    - name: otlp
      port: 4317
      targetPort: 4317
      protocol: TCP
      appProtocol: grpc
    - name: otlp-http
      port: 4318
      targetPort: 4318
      protocol: TCP
    - name: zipkin
      port: 9411
      targetPort: 9411
      protocol: TCP
  selector:
    app.kubernetes.io/name: opentelemetry-collector
    app.kubernetes.io/instance: opentelemetry-collector
    component: standalone-collector
  internalTrafficPolicy: Cluster
---
# Source: opentelemetry-collector/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: opentelemetry-collector
  namespace: grafana-stack
  labels:
    helm.sh/chart: opentelemetry-collector-0.106.0
    app.kubernetes.io/name: opentelemetry-collector
    app.kubernetes.io/instance: opentelemetry-collector
    app.kubernetes.io/version: "0.109.0"
    app.kubernetes.io/managed-by: Helm

spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/name: opentelemetry-collector
      app.kubernetes.io/instance: opentelemetry-collector
      component: standalone-collector
  strategy:
    type: RollingUpdate
  template:
    metadata:
      annotations:
        checksum/config: 950dd817a4cf0b9376662c7d3fea1249b296640ba4685e6b247de5dbcf78f562

      labels:
        app.kubernetes.io/name: opentelemetry-collector
        app.kubernetes.io/instance: opentelemetry-collector
        component: standalone-collector

    spec:

      serviceAccountName: opentelemetry-collector
      securityContext:
        {}
      containers:
        - name: opentelemetry-collector
          args:
            - --config=/conf/relay.yaml
          securityContext:
            {}
          image: "otel/opentelemetry-collector-k8s:0.109.0"
          imagePullPolicy: IfNotPresent
          ports:

            - name: jaeger-compact
              containerPort: 6831
              protocol: UDP
            - name: jaeger-grpc
              containerPort: 14250
              protocol: TCP
            - name: jaeger-thrift
              containerPort: 14268
              protocol: TCP
            - name: otlp
              containerPort: 4317
              protocol: TCP
            - name: otlp-http
              containerPort: 4318
              protocol: TCP
            - name: zipkin
              containerPort: 9411
              protocol: TCP
          env:
            - name: MY_POD_IP
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: status.podIP
          livenessProbe:
            httpGet:
              path: /
              port: 13133
          readinessProbe:
            httpGet:
              path: /
              port: 13133
          volumeMounts:
            - mountPath: /conf
              name: opentelemetry-collector-configmap
      volumes:
        - name: opentelemetry-collector-configmap
          configMap:
            name: opentelemetry-collector
            items:
              - key: relay
                path: relay.yaml
      hostNetwork: false

zking2000 commented 4 days ago

Flask
Werkzeug
opentelemetry-distro
opentelemetry-exporter-otlp
requests
opentelemetry-instrumentation-flask
protobuf

zking2000 commented 4 days ago

from opentelemetry.instrumentation.flask import FlaskInstrumentor
from opentelemetry.instrumentation.psycopg2 import Psycopg2Instrumentor
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry import trace

# 设置OTLP导出器
otlp_exporter = OTLPSpanExporter(endpoint="your-otlp-endpoint")
trace.set_tracer_provider(TracerProvider())
trace.get_tracer_provider().add_span_processor(BatchSpanProcessor(otlp_exporter))

# 启用自动插装
FlaskInstrumentor().instrument()
Psycopg2Instrumentor().instrument()

# 您的Flask应用程序代码
from flask import Flask
app = Flask(__name__)
# ... 其余的应用程序代码 ...

zking2000 commented 4 days ago


@app.route('/user', methods=['GET'])
def get_user():
    tracer = trace.get_tracer(__name__)
    with tracer.start_as_current_span("get_user_database_query"):
        username = request.args.get('username')
        if not username:
            return jsonify({'error': 'Username is required'}), 400

        try:
            conn = psycopg2.connect(**db_config)
            cursor = conn.cursor(cursor_factory=RealDictCursor)

            # New tracing span for the database operation
            with tracer.start_as_current_span("execute_database_query"):
                query = "SELECT * FROM users WHERE username = %s"
                cursor.execute(query, (username,))
                user_data = cursor.fetchone()

            cursor.close()
            conn.close()

            if user_data:
                return jsonify(user_data)
            else:
                return jsonify({'error': 'User not found'}), 404
        except psycopg2.Error as err:
            return jsonify({'error': str(err)}), 500

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)

zking2000 commented 4 days ago


@app.route('/user', methods=['GET'])
def get_user():
    tracer = trace.get_tracer(__name__)
    with tracer.start_as_current_span("get_user_database_query"):
        username = request.args.get('username')
        if not username:
            return jsonify({'error': 'Username is required'}), 400

        try:
            conn = psycopg2.connect(**db_config)
            cursor = conn.cursor(cursor_factory=RealDictCursor)

            # Detailed tracing for database operations
            with tracer.start_as_current_span("database_connection") as conn_span:
                conn_span.set_attribute("db.system", "postgresql")
                conn_span.set_attribute("db.name", db_config['database'])
                conn_span.set_attribute("db.user", db_config['user'])
                conn_span.set_attribute("net.peer.name", db_config['host'])
                conn_span.set_attribute("net.peer.port", db_config['port'])

            query = "SELECT * FROM users WHERE username = %s"
            with tracer.start_as_current_span("execute_query") as query_span:
                query_span.set_attribute("db.statement", query)
                query_span.set_attribute("db.operation", "SELECT")
                cursor.execute(query, (username,))

            with tracer.start_as_current_span("fetch_results"):
                user_data = cursor.fetchone()

            with tracer.start_as_current_span("close_database_resources"):
                cursor.close()
                conn.close()

            if user_data:
                return jsonify(user_data)
            else:
                return jsonify({'error': 'User not found'}), 404
        except psycopg2.Error as err:
            with tracer.start_as_current_span("database_error"):
                return jsonify({'error': str(err)}), 500

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)

zking2000 commented 4 days ago


# Tempo configuration file

server:
  http_listen_port: 3200

distributor:
  receivers:
    jaeger:
      protocols:
        thrift_http:
          endpoint: 0.0.0.0:14268

storage:
  trace:
    backend: local
    local:
      path: /tmp/tempo/traces

# Service Graphs configuration
service_graphs:
  enabled: true

# Additional Service Graphs settings (optional)
# max_items: 10000
# wait: 5s
# backend: local
# local:
#   path: /tmp/tempo/service_graphs

# If using Prometheus as backend:
# backend: prometheus
# prometheus:
#   endpoint: http://prometheus:9090
#   max_nodes: 100

zking2000 commented 3 days ago

了解您的指标存储在Mimir中，这确实会改变一些配置和检查步骤。Mimir是一个高性能的指标存储系统，通常用作Prometheus的长期存储解决方案。让我们根据这个情况调整检查和配置步骤：

确保数据正确写入Mimir：

a. 检查Prometheus配置：确保Prometheus配置为将数据远程写入Mimir。在Prometheus的配置文件中应该有类似这样的设置：
```
  remote_write:
    - url: "http://mimir:9009/api/v1/push"
```
b. 验证Prometheus到Mimir的连接：检查Prometheus的日志，确保没有与远程写入相关的错误。
直接从Mimir查询数据：

a. 使用Mimir的查询接口（通常与Prometheus兼容）来验证数据是否存在。可以通过curl或类似工具发送HTTP请求：
```
  curl -G 'http://mimir:9009/prometheus/api/v1/query' --data-urlencode 'query=request_count'
```
b. 如果您有Grafana设置，将其配置为直接从Mimir读取数据。在Grafana的数据源设置中，添加Mimir作为Prometheus类型的数据源。
检查指标和标签：

使用Mimir的查询接口或Grafana，运行以下查询来检查数据：
```
request_count
rate(request_count[5m])
histogram_quantile(0.95, rate(request_latency_seconds_bucket[5m]))
```
确保这些查询返回预期的结果，包含正确的标签（source和destination）。
服务图配置：

a. 如果您使用Grafana的服务图插件，确保它配置为使用Mimir数据源。

b. 服务图可能需要特定的指标格式。检查您的服务图工具文档，确保生成的指标符合其要求。
故障排除：

a. 检查Mimir的日志，寻找任何接收或存储数据时的错误。

b. 验证Mimir的保留策略，确保数据没有被过早删除。

c. 如果使用Grafana，检查其日志中是否有与Mimir数据源连接相关的错误。
数据一致性检查：

a. 比较Prometheus和Mimir中的数据，确保数据正确传输和存储。

b. 检查时间戳，确保没有时间同步问题导致的数据丢失。
生成测试数据：

使用之前提供的Python脚本持续生成测试数据，但确保Prometheus配置正确地将这些数据写入Mimir。
Mimir特定的查询优化：

Mimir可能支持一些特定的查询优化或特性。查阅Mimir文档，了解是否有任何可以优化服务图查询性能的特定设置。

如果执行这些步骤后服务图仍然没有数据，可能需要更深入地检查：

Mimir的具体版本和配置。
用于生成服务图的具体工具或Grafana插件的详细信息。
任何自定义的查询或转换逻辑，可能影响服务图的数据处理。

提供这些额外信息将有助于进一步诊断问题并提供更具体的解决方案。

zking2000 commented 3 days ago

OpenTelemetry 确实可以生成 span metrics，但这通常需要额外的配置。让我们逐步检查和配置：

检查当前 metrics：首先，确认您当前的 OpenTelemetry 配置是否已经在生成 span metrics。

a. 在 Grafana 或 Prometheus/Mimir 中，查找类似以下名称的指标：
- spans_total
- span_duration_milliseconds
- calls_total
b. 检查这些指标是否包含服务名称等标签。

如果不包含 span metrics，配置 OpenTelemetry 以生成它们：

a. 在您的 OpenTelemetry Collector 配置中，添加或修改 spanmetrics processor：

  processors:
    spanmetrics:
      metrics_exporter: prometheus
      latency_histogram_buckets: [1ms, 2ms, 6ms, 10ms, 100ms, 250ms]
      dimensions:
        - name: service.name
          default: unknown_service
        - name: operation
          default: unknown_operation
      dimensions_cache_size: 1000
      aggregation_temporality: AGGREGATION_TEMPORALITY_CUMULATIVE

b. 在 pipelines 配置中启用 spanmetrics processor：

  service:
    pipelines:
      traces:
        processors: [..., spanmetrics, ...]
      metrics:
        receivers: [otlp]
        processors: [batch]
        exporters: [prometheus]

配置 Prometheus/Mimir 抓取：确保您的 Prometheus 或 Mimir 配置为从 OpenTelemetry Collector 抓取指标：
```
scrape_configs:
 - job_name: 'otel-collector'
   static_configs:
     - targets: ['otel-collector:8888']
```
重启 OpenTelemetry Collector：应用新的配置后，重启 Collector 以生效。
验证 span metrics：在 Grafana 或 Prometheus/Mimir 中查询新生成的 span metrics：
```
spans_total
span_duration_milliseconds
calls_total
```
在 Grafana 中创建 Service Graph： a. 进入 Explore 页面。 b. 选择 Prometheus/Mimir 数据源。 c. 使用如下查询：
```
  sum(rate(calls_total{}[5m])) by (service_name, operation)
```
d. 选择 "Node Graph" 可视化。 e. 配置 "Source" 为 service_name，"Target" 为 operation。
优化和调整：
- 调整时间范围以获取足够的数据。
- 考虑添加额外的维度，如 client.service.name 和 server.service.name。
- 根据需要调整可视化设置。
故障排除：
- 如果没有看到数据，检查 OpenTelemetry Collector 的日志。
- 确保 Prometheus/Mimir 成功抓取了指标。
- 验证应用程序是否正确发送追踪数据到 Collector。
高级配置：
- 考虑使用 OpenTelemetry 的自动检测（auto-instrumentation）来捕获更多详细的追踪数据。
- 探索使用 exemplars 来链接指标和追踪。
持续优化：
- 监控 span metrics 的性能影响。
- 根据需要调整采样率和保留策略。

如果在执行这些步骤时遇到任何问题，请提供以下信息：

OpenTelemetry Collector 的版本
当前的 Collector 配置（如果可能）
任何错误消息或意外行为
示例指标数据

有了这些信息，我可以提供更具体的指导来帮助您成功配置和使用 span metrics 创建 Service Graph。

zking2000 commented 3 days ago

当然，我可以为您写一个 shell 脚本来同时重启 OpenTelemetry Agent (otel-agent) 和 Nginx。这里是一个示例脚本：

#!/bin/bash

# 定义颜色代码
GREEN='\033[0;32m'
RED='\033[0;31m'
NC='\033[0m' # No Color

# 定义重启函数
restart_service() {
    service_name=$1
    echo -e "${GREEN}Restarting $service_name...${NC}"
    if systemctl is-active --quiet $service_name; then
        sudo systemctl restart $service_name
        if [ $? -eq 0 ]; then
            echo -e "${GREEN}$service_name restarted successfully.${NC}"
        else
            echo -e "${RED}Failed to restart $service_name.${NC}"
            exit 1
        fi
    else
        echo -e "${RED}$service_name is not running. Starting it...${NC}"
        sudo systemctl start $service_name
        if [ $? -eq 0 ]; then
            echo -e "${GREEN}$service_name started successfully.${NC}"
        else
            echo -e "${RED}Failed to start $service_name.${NC}"
            exit 1
        fi
    fi
}

# 重启 otel-agent
restart_service otel-agent

# 重启 nginx
restart_service nginx

echo -e "${GREEN}Both otel-agent and nginx have been restarted.${NC}"

使用说明：

将这个脚本保存为文件，例如 restart_services.sh。
给脚本添加执行权限：
```
chmod +x restart_services.sh
```
运行脚本（需要 sudo 权限）：
```
sudo ./restart_services.sh
```

这个脚本做以下几件事：

定义了颜色代码，用于输出彩色文本，使信息更易读。
创建了一个通用的 restart_service 函数，可以重启任何系统服务。
分别重启 otel-agent 和 nginx 服务。
如果服务当前没有运行，脚本会尝试启动它。
在每次操作后检查是否成功，如果失败则输出错误信息并退出。

注意事项：

这个脚本假设 otel-agent 和 nginx 都是作为 systemd 服务运行的。如果您的系统使用不同的服务管理器，可能需要相应地调整脚本。
脚本需要 sudo 权限来重启系统服务。
确保在运行脚本之前，服务名称是正确的。某些系统中，OpenTelemetry Agent 的服务名可能不是 otel-agent，可能是 otelcol 或其他名称。请根据您的系统配置进行调整。

如果您需要针对特定环境进行任何调整，请告诉我，我会很乐意帮您修改脚本。

zking2000 commented 3 days ago

我非常感谢您的持续反馈。看来 OpenTelemetry Collector 的变化比预期的更加频繁和剧烈。我为之前的不准确信息再次道歉。

既然 spanmetrics 和 servicegraph 都不再可用，我们需要采取一个更基本的方法来生成服务依赖图所需的指标。我们可以使用 OpenTelemetry Collector 的基本功能来处理 spans 并生成所需的指标。

以下是一个可能的替代方案：

使用 groupbytrace 和 metricstransform 处理器

receivers:
  otlp:
    protocols:
      grpc:
      http:

processors:
  batch:
  groupbytrace:
    wait_duration: 1s
    num_traces: 100
  metricstransform:
    transforms:
      - include: calls_total
        action: insert
        new_name: service_calls_total
        operations:
          - action: aggregate_labels
            aggregation_type: sum
            label_set: [ "service.name", "destination.service.name" ]

exporters:
  prometheus:
    endpoint: "0.0.0.0:8889"

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch, groupbytrace, metricstransform]
      exporters: [prometheus]

这个配置做以下几件事：

使用 groupbytrace 处理器来组织相关的 spans。
使用 metricstransform 处理器来创建和转换指标。
生成一个名为 service_calls_total 的指标，它包含 service.name 和 destination.service.name 标签。

要使用这个配置：

更新您的 OpenTelemetry Collector 配置文件。
重启 Collector 服务。
检查 Collector 日志以确保没有错误。
访问 Prometheus 导出端点（:8889）验证新指标是否正在生成。
更新您的 Grafana 仪表板以使用这些新生成的指标。

请注意，这个解决方案可能需要根据您的具体需求进行调整。它不会直接生成完整的服务图，但会提供构建服务图所需的基本指标。

如果这个方法仍然不适用于您的环境，或者您遇到了其他问题，请提供以下信息：

您正在使用的 OpenTelemetry Collector 的确切版本。
您希望在服务图中看到的具体信息或指标。
您当前的 OpenTelemetry Collector 配置文件的内容。
任何错误消息或意外行为。

有了这些信息，我们可以一起找到一个更适合您特定需求和环境的解决方案。再次感谢您的耐心和反馈，这有助于我们提供更准确和最新的建议。

zking2000 commented 3 days ago

Here's the Python code shown in the image:

from prometheus_client import start_http_server, Summary, Counter, Gauge, Histogram
import random
import time

# Metrics to be collected
REQUESTS = Counter('requests_total', 'Total number of requests')
ERRORS = Counter('errors_total', 'Total number of errors')
RESPONSE_TIME = Summary('response_time_seconds', 'Response time in seconds')
DEPENDENCIES = Gauge('service_dependencies', 'Number of dependencies per service', ['service_name'])
SATURATION = Gauge('service_saturation', 'Resource saturation level', ['resource_type'])
RETRIES = Counter('retries_total', 'Total number of retries')
TRAFFIC = Histogram('traffic_distribution', 'Distribution of traffic between services', ['service_name'])

# Example function to simulate request handling
def handle_request():
    REQUESTS.inc()
    if random.random() < 0.2:  # Simulate errors
        ERRORS.inc()
        return
    start = time.time()
    time.sleep(random.random())  # Simulate response time
    RESPONSE_TIME.observe(time.time() - start)
    # Simulate dependencies, saturation, retries, and traffic distribution
    DEPENDENCIES.labels(service_name='serviceX').set(3)
    SATURATION.labels(resource_type='CPU').set(0.75)
    RETRIES.inc()
    TRAFFIC.labels(service_name='serviceY').observe(1)

if __name__ == '__main__':
    start_http_server(9999)  # Start a server to expose metrics
    while True:
        handle_request()
        time.sleep(1)

This code sets up a Prometheus metrics collection system for a simulated service, including various metrics like request counts, errors, response times, and more. It then defines a handle_request() function to simulate handling requests and updating these metrics. Finally, it starts a server on port 9999 to expose the metrics and enters a loop to continuously simulate request handling.