gohangout 处理嵌套数据不成功

[KevinLiangX]

消息类型： "kubernetes": { "namespace": "default", "statefulset": { "name": "clickhouse-shard0" }, "labels": { "helm_sh/chart": "clickhouse-3.2.1", "statefulset_kubernetes_io/pod-name": "clickhouse-shard0-1", "app_kubernetes_io/component": "clickhouse", "app_kubernetes_io/instance": "clickhouse", "app_kubernetes_io/managed-by": "Helm", "app_kubernetes_io/name": "clickhouse", "controller-revision-hash": "clickhouse-shard0-5cb5bf8df5" }, "container": { "name": "clickhouse", "image": "docker.io/bitnami/clickhouse:23.3.2-debian-11-r0" }, "node": { "name": "host01" }, "pod": { "name": "clickhouse-shard0-1", "uid": "0c1743a6-4c67-4ed3-90aa-2c74545edb93" } }

inputs:
    - Kafka:
        topic:
            filebeat: 1
        codec: json
        consumer_settings:
            bootstrap.servers: "10.96.3.252:9092"
            group.id: gohangout.filebeat
filters:
    - Add:
        overwrite: true
        fields:
            pod_name: "%{[kubernetes][pod][name]}"
            namespace: '$.kubernetes.namespace'
            container_name: '$.kubernetes.container.name'
            node_name: '$.kubernetes.node'
outputs:
    - Clickhouse:
        table: 'logs.log_queue_all'
        conn_max_life_time: 1800
        username: admin
        password: Fh123456
        hosts:
        - 'tcp://10.96.0.120:9000'
        fields: ['pod_name','namespace','container_name','node_name','message']
        auto_convert: false
        bulk_actions: 1000
        flush_interval: 30
        concurrent: 1

clickhouse 表中有 pod_name, namespace ,container_name,node_name等列

使用了jsonPath方式调用数据，和%访问，通过添加一个新的fields 去匹配数据库的列名称，没有生效，这种嵌套的数据该如何处理？

[KevinLiangX]

问题点在 fields中的 顺序 不能乱，要跟后面表的列结构 保持顺序一致

[childe]

nodename 哪里的 jsonpath 表达式少写了一个 .name ？

发自我的iPhone

------------------ 原始邮件 ------------------ 发件人: KevinLiangX @.> 发送时间: 2023年5月6日 14:35 收件人: childe/gohangout @.> 抄送: Subscribed @.***> 主题: Re: [childe/gohangout] gohangout 处理嵌套数据不成功 (Issue #216)

消息类型： "kubernetes": { "namespace": "default", "statefulset": { "name": "clickhouse-shard0" }, "labels": { "helm_sh/chart": "clickhouse-3.2.1", "statefulset_kubernetes_io/pod-name": "clickhouse-shard0-1", "app_kubernetes_io/component": "clickhouse", "app_kubernetes_io/instance": "clickhouse", "app_kubernetes_io/managed-by": "Helm", "app_kubernetes_io/name": "clickhouse", "controller-revision-hash": "clickhouse-shard0-5cb5bf8df5" }, "container": { "name": "clickhouse", "image": "docker.io/bitnami/clickhouse:23.3.2-debian-11-r0" }, "node": { "name": "host01" }, "pod": { "name": "clickhouse-shard0-1", "uid": "0c1743a6-4c67-4ed3-90aa-2c74545edb93" } } inputs: - Kafka: topic: filebeat: 1 codec: json consumer_settings: bootstrap.servers: "10.96.3.252:9092" group.id: gohangout.filebeat filters: - Add: overwrite: true fields: pod_name: "%{[kubernetes][pod][name]}" namespace: '$.kubernetes.namespace' container_name: '$.kubernetes.container.name' node_name: '$.kubernetes.node' outputs: - Clickhouse: table: 'logs.log_queue_all' conn_max_life_time: 1800 username: admin password: Fh123456 hosts: - 'tcp://10.96.0.120:9000' fields: ['pod_name','namespace','container_name','node_name','message'] auto_convert: false bulk_actions: 1000 flush_interval: 30 concurrent: 1
clickhouse 表中有 pod_name, namespace ,container_name,node_name等列

使用了jsonPath方式调用数据，和%访问，通过添加一个新的fields 去匹配数据库的列名称，没有生效，这种嵌套的数据该如何处理？

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.***>