search

aws / aws-for-fluent-bit

The source of the amazon/aws-for-fluent-bit container image
Apache License 2.0
437 stars 130 forks source link

consistent error: caught signal (SIGSEGV) #816

Open June3Ningxu opened 1 month ago

June3Ningxu commented 1 month ago

hi all, Fluentbit version: amazon/aws-for-fluent-bit:debug-2.32.2.20240516 Deployment mode: AWS ECS Fargate sidecar Programming language: N/A Log format: JSON

I have an issue when running fluentbit in my AWS ECS cluster. It consistently shows caught signal(SIGSEGV) error, and I checked the error code from fluent-bit container is 139.

CoreDump Log updated

issue_2024-06-20T190044_host-ip-192-168-194-12.cn-north-1.compute.internal_101311429628425.all.zip

issue_2024-06-21T000043_host-ip-192-168-65-99.cn-north-1.compute.internal_7619239412270.all.zip

Here is the log

Fluent Bit Log Output

Fluent Bit v1.9.10
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io
[2024/05/13 07:40:59] [ info] [fluent bit] version=1.9.10, commit=9be1f19e5a, pid=1
[2024/05/13 07:40:59] [ info] [storage] version=1.4.0, type=memory-only, sync=normal, checksum=disabled, max_chunks_up=128
[2024/05/13 07:40:59] [ info] [cmetrics] version=0.3.7
[2024/05/13 07:40:59] [ info] [input:tcp:tcp.0] listening on 127.0.0.1:8877
[2024/05/13 07:40:59] [ info] [input:forward:forward.1] listening on unix:///var/run/fluent.sock
[2024/05/13 07:40:59] [ info] [input:forward:forward.2] listening on 127.0.0.1:24224
[2024/05/13 07:40:59] [ info] [output:null:null.0] worker #0 started
[2024/05/13 07:40:59] [ info] [output:loki:loki.1] configured, hostname=192.168.133.167:3100
[2024/05/13 07:40:59] [ info] [sp] stream processor started
[2024/05/13 07:41:58] [engine] caught signal (SIGSEGV)
AWS for Fluent Bit Container Image Version 2.32.2.20240425`

Configuration

This is the ecs task definition configuration:

"logConfiguration": {
                "logDriver": "awsfirelens",
                "options": {
                    "Host": "192.168.133.167",
                    "Name": "loki",
                    "label_keys": "$container_name,$ecs_task_definition,$source,$ecs_cluster",
                    "labels": "job=firelens",
                    "line_format": "key_value",
                    "port": "3100",
                    "remove_keys": "container_id,ecs_task_arn"
                }
            },

{
            "name": "log_router",
            "image": "public.ecr.aws/aws-observability/aws-for-fluent-bit:2.32.2.20240425",
            "cpu": 0,
            "memoryReservation": 500,
            "portMappings": [],
            "essential": false,
            "environment": [],
            "mountPoints": [],
            "volumesFrom": [],
            "user": "0",
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-create-group": "true",
                    "awslogs-group": "/ecs/ecs-aws-firelens-sidecar-container",
                    "awslogs-region": "cn-north-1",
                    "awslogs-stream-prefix": "firelens"
                }
            },
            "systemControls": [],
            "firelensConfiguration": {
                "type": "fluentbit",
                "options": {
                    "enable-ecs-log-metadata": "true"
                }
            }
        }
    ],

Updated 2024/06/19

after enable the debug log as below, the log shows no any error also

"environment": [
                {
                    "name": "FLB_LOG_LEVEL",
                    "value": "debug"
                }
            ],

Log

Fluent Bit v1.9.10
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io
[2024/05/28 06:49:10] [ info] Configuration:
[2024/05/28 06:49:10] [ info]  flush time     | 1.000000 seconds
[2024/05/28 06:49:10] [ info]  grace          | 5 seconds
[2024/05/28 06:49:10] [ info]  daemon         | 0
[2024/05/28 06:49:10] [ info] ___________
[2024/05/28 06:49:10] [ info]  inputs:
[2024/05/28 06:49:10] [ info]      tcp
[2024/05/28 06:49:10] [ info]      forward
[2024/05/28 06:49:10] [ info]      forward
[2024/05/28 06:49:10] [ info] ___________
[2024/05/28 06:49:10] [ info]  filters:
[2024/05/28 06:49:10] [ info]      record_modifier.0
[2024/05/28 06:49:10] [ info] ___________
[2024/05/28 06:49:10] [ info]  outputs:
[2024/05/28 06:49:10] [ info]      null.0
[2024/05/28 06:49:10] [ info]      loki.1
[2024/05/28 06:49:10] [ info] ___________
[2024/05/28 06:49:10] [ info]  collectors:
[2024/05/28 06:49:10] [ info] [fluent bit] version=1.9.10, commit=9be1f19e5a, pid=1
[2024/05/28 06:49:10] [debug] [engine] coroutine stack size: 24576 bytes (24.0K)
[2024/05/28 06:49:10] [ info] [storage] version=1.4.0, type=memory-only, sync=normal, checksum=disabled, max_chunks_up=128
[2024/05/28 06:49:10] [ info] [cmetrics] version=0.3.7
[2024/05/28 06:49:10] [debug] [tcp:tcp.0] created event channels: read=27 write=28
[2024/05/28 06:49:10] [ info] [input:tcp:tcp.0] listening on 127.0.0.1:8877
[2024/05/28 06:49:10] [debug] [forward:forward.1] created event channels: read=30 write=31
[2024/05/28 06:49:10] [ info] [input:forward:forward.1] listening on unix:///var/run/fluent.sock
[2024/05/28 06:49:10] [debug] [forward:forward.2] created event channels: read=33 write=34
[2024/05/28 06:49:10] [debug] [in_fw] Listen='127.0.0.1' TCP_Port=24224
[2024/05/28 06:49:10] [ info] [input:forward:forward.2] listening on 127.0.0.1:24224
[2024/05/28 06:49:10] [debug] [null:null.0] created event channels: read=36 write=37
[2024/05/28 06:49:10] [debug] [loki:loki.1] created event channels: read=43 write=44
[2024/05/28 06:49:10] [ info] [output:null:null.0] worker #0 started
[2024/05/28 06:49:10] [debug] [output:loki:loki.1] remove_mpa size: 6
[2024/05/28 06:49:10] [ info] [output:loki:loki.1] configured, hostname=192.168.133.167:3100
[2024/05/28 06:49:10] [debug] [router] match rule tcp.0:null.0
[2024/05/28 06:49:10] [ info] [sp] stream processor started
[2024/05/28 06:49:11] [debug] [input chunk] update output instances with new chunk size diff=285, records=1, input=forward.1
[2024/05/28 06:49:11] [debug] [input chunk] update output instances with new chunk size diff=325, records=1, input=forward.1
[2024/05/28 06:49:11] [debug] [input chunk] update output instances with new chunk size diff=326, records=1, input=forward.1
[2024/05/28 06:49:11] [debug] [input chunk] update output instances with new chunk size diff=327, records=1, input=forward.1
[2024/05/28 06:49:11] [debug] [input chunk] update output instances with new chunk size diff=328, records=1, input=forward.1
[2024/05/28 06:49:11] [debug] [input chunk] update output instances with new chunk size diff=327, records=1, input=forward.1
[2024/05/28 06:49:11] [debug] [input chunk] update output instances with new chunk size diff=326, records=1, input=forward.1
[2024/05/28 06:49:11] [debug] [input chunk] update output instances with new chunk size diff=328, records=1, input=forward.1
[2024/05/28 06:49:11] [debug] [input chunk] update output instances with new chunk size diff=285, records=1, input=forward.1
[2024/05/28 06:49:11] [debug] [input chunk] update output instances with new chunk size diff=582, records=1, input=forward.1
[2024/05/28 06:49:11] [debug] [input chunk] update output instances with new chunk size diff=459, records=1, input=forward.1
[2024/05/28 06:49:11] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:11] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:11] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:11] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:11] [debug] [out flush] cb_destroy coro_id=0
[2024/05/28 06:49:11] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:13] [debug] [input chunk] update output instances with new chunk size diff=436, records=1, input=forward.1
[2024/05/28 06:49:13] [debug] [input chunk] update output instances with new chunk size diff=417, records=1, input=forward.1
[2024/05/28 06:49:13] [debug] [input chunk] update output instances with new chunk size diff=439, records=1, input=forward.1
[2024/05/28 06:49:13] [debug] [input chunk] update output instances with new chunk size diff=442, records=1, input=forward.1
[2024/05/28 06:49:13] [debug] [input chunk] update output instances with new chunk size diff=465, records=1, input=forward.1
[2024/05/28 06:49:13] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:13] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:13] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:13] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:13] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:13] [debug] [out flush] cb_destroy coro_id=1
[2024/05/28 06:49:13] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:14] [debug] [input chunk] update output instances with new chunk size diff=492, records=1, input=forward.1
[2024/05/28 06:49:14] [debug] [input chunk] update output instances with new chunk size diff=460, records=1, input=forward.1
[2024/05/28 06:49:14] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:14] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:14] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:14] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:14] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:14] [debug] [out flush] cb_destroy coro_id=2
[2024/05/28 06:49:14] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:14] [debug] [input chunk] update output instances with new chunk size diff=316, records=1, input=forward.1
[2024/05/28 06:49:14] [debug] [input chunk] update output instances with new chunk size diff=316, records=1, input=forward.1
[2024/05/28 06:49:14] [debug] [input chunk] update output instances with new chunk size diff=316, records=1, input=forward.1
[2024/05/28 06:49:14] [debug] [input chunk] update output instances with new chunk size diff=315, records=1, input=forward.1
[2024/05/28 06:49:15] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:15] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:15] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:15] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:15] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:15] [debug] [out flush] cb_destroy coro_id=3
[2024/05/28 06:49:15] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:17] [debug] [input chunk] update output instances with new chunk size diff=572, records=1, input=forward.1
[2024/05/28 06:49:17] [debug] [input chunk] update output instances with new chunk size diff=457, records=1, input=forward.1
[2024/05/28 06:49:17] [debug] [input chunk] update output instances with new chunk size diff=566, records=1, input=forward.1
[2024/05/28 06:49:17] [debug] [input chunk] update output instances with new chunk size diff=455, records=1, input=forward.1
[2024/05/28 06:49:17] [debug] [input chunk] update output instances with new chunk size diff=575, records=1, input=forward.1
[2024/05/28 06:49:17] [debug] [input chunk] update output instances with new chunk size diff=458, records=1, input=forward.1
[2024/05/28 06:49:18] [debug] [input chunk] update output instances with new chunk size diff=444, records=1, input=forward.1
[2024/05/28 06:49:18] [debug] [input chunk] update output instances with new chunk size diff=436, records=1, input=forward.1
[2024/05/28 06:49:18] [debug] [input chunk] update output instances with new chunk size diff=419, records=1, input=forward.1
[2024/05/28 06:49:18] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:18] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:18] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:18] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:18] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:18] [debug] [input chunk] update output instances with new chunk size diff=425, records=1, input=forward.1
[2024/05/28 06:49:18] [debug] [out flush] cb_destroy coro_id=4
[2024/05/28 06:49:18] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:18] [debug] [input chunk] update output instances with new chunk size diff=451, records=1, input=forward.1
[2024/05/28 06:49:18] [debug] [input chunk] update output instances with new chunk size diff=467, records=1, input=forward.1
[2024/05/28 06:49:19] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:19] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:19] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:19] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:19] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:19] [debug] [out flush] cb_destroy coro_id=5
[2024/05/28 06:49:19] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:25] [debug] [input chunk] update output instances with new chunk size diff=449, records=1, input=forward.1
[2024/05/28 06:49:25] [debug] [input chunk] update output instances with new chunk size diff=432, records=1, input=forward.1
[2024/05/28 06:49:25] [debug] [input chunk] update output instances with new chunk size diff=424, records=1, input=forward.1
[2024/05/28 06:49:26] [debug] [input chunk] update output instances with new chunk size diff=418, records=1, input=forward.1
[2024/05/28 06:49:26] [debug] [input chunk] update output instances with new chunk size diff=423, records=1, input=forward.1
[2024/05/28 06:49:26] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:26] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:26] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:26] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:26] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:26] [debug] [out flush] cb_destroy coro_id=6
[2024/05/28 06:49:26] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:31] [debug] [input chunk] update output instances with new chunk size diff=14701, records=1, input=forward.1
[2024/05/28 06:49:32] [debug] [input chunk] update output instances with new chunk size diff=966, records=1, input=forward.1
[2024/05/28 06:49:32] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:32] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:32] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:32] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:32] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:32] [debug] [out flush] cb_destroy coro_id=7
[2024/05/28 06:49:32] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:32] [debug] [input chunk] update output instances with new chunk size diff=961, records=1, input=forward.1
[2024/05/28 06:49:33] [debug] [input chunk] update output instances with new chunk size diff=936, records=1, input=forward.1
[2024/05/28 06:49:33] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:33] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:33] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:33] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:33] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:33] [debug] [out flush] cb_destroy coro_id=8
[2024/05/28 06:49:33] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:34] [debug] [input chunk] update output instances with new chunk size diff=985, records=1, input=forward.1
[2024/05/28 06:49:35] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:35] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:35] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:35] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:35] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:35] [debug] [out flush] cb_destroy coro_id=9
[2024/05/28 06:49:35] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:35] [debug] [input chunk] update output instances with new chunk size diff=14700, records=1, input=forward.1
[2024/05/28 06:49:36] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:36] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:36] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:36] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:36] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:36] [debug] [out flush] cb_destroy coro_id=10
[2024/05/28 06:49:36] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:36] [debug] [input chunk] update output instances with new chunk size diff=908, records=1, input=forward.1
[2024/05/28 06:49:37] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:37] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:37] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:37] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:37] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:37] [debug] [out flush] cb_destroy coro_id=11
[2024/05/28 06:49:37] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:38] [debug] [input chunk] update output instances with new chunk size diff=14699, records=1, input=forward.1
[2024/05/28 06:49:39] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:39] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:39] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:39] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:39] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:39] [debug] [out flush] cb_destroy coro_id=12
[2024/05/28 06:49:39] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:40] [debug] [input chunk] update output instances with new chunk size diff=14699, records=1, input=forward.1
[2024/05/28 06:49:41] [debug] [input chunk] update output instances with new chunk size diff=972, records=1, input=forward.1
[2024/05/28 06:49:41] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:41] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:41] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:41] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:41] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:41] [debug] [out flush] cb_destroy coro_id=13
[2024/05/28 06:49:41] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:42] [debug] [input chunk] update output instances with new chunk size diff=14699, records=1, input=forward.1
[2024/05/28 06:49:43] [debug] [input chunk] update output instances with new chunk size diff=14699, records=1, input=forward.1
[2024/05/28 06:49:43] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:43] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:43] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:43] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:43] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:43] [debug] [out flush] cb_destroy coro_id=14
[2024/05/28 06:49:43] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:45] [debug] [input chunk] update output instances with new chunk size diff=963, records=1, input=forward.1
[2024/05/28 06:49:45] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:45] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:45] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:45] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:45] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:45] [debug] [out flush] cb_destroy coro_id=15
[2024/05/28 06:49:45] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:46] [debug] [input chunk] update output instances with new chunk size diff=14699, records=1, input=forward.1
[2024/05/28 06:49:46] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:46] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:46] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:46] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:46] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:46] [debug] [out flush] cb_destroy coro_id=16
[2024/05/28 06:49:46] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:48] [debug] [input chunk] update output instances with new chunk size diff=14699, records=1, input=forward.1
[2024/05/28 06:49:49] [debug] [input chunk] update output instances with new chunk size diff=14699, records=1, input=forward.1
[2024/05/28 06:49:49] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:49] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:49] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:49] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:49] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:49] [debug] [out flush] cb_destroy coro_id=17
[2024/05/28 06:49:49] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:49] [debug] [input chunk] update output instances with new chunk size diff=14699, records=1, input=forward.1
[2024/05/28 06:49:50] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:50] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:50] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:50] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:50] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:50] [debug] [out flush] cb_destroy coro_id=18
[2024/05/28 06:49:50] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:51] [debug] [input chunk] update output instances with new chunk size diff=14699, records=1, input=forward.1
[2024/05/28 06:49:52] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:52] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:52] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:52] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:52] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:52] [debug] [out flush] cb_destroy coro_id=19
[2024/05/28 06:49:52] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:52] [debug] [input chunk] update output instances with new chunk size diff=963, records=1, input=forward.1
[2024/05/28 06:49:53] [debug] [input chunk] update output instances with new chunk size diff=14699, records=1, input=forward.1
[2024/05/28 06:49:53] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:53] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:53] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:53] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:53] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:53] [debug] [out flush] cb_destroy coro_id=20
[2024/05/28 06:49:53] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:55] [debug] [input chunk] update output instances with new chunk size diff=957, records=1, input=forward.1
[2024/05/28 06:49:55] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:55] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:55] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:55] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:55] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:55] [debug] [out flush] cb_destroy coro_id=21
[2024/05/28 06:49:55] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:56] [debug] [input chunk] update output instances with new chunk size diff=14699, records=1, input=forward.1
[2024/05/28 06:49:57] [debug] [input chunk] update output instances with new chunk size diff=14699, records=1, input=forward.1
[2024/05/28 06:49:57] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:57] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:57] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:57] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:57] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:57] [debug] [out flush] cb_destroy coro_id=22
[2024/05/28 06:49:57] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:49:58] [debug] [input chunk] update output instances with new chunk size diff=966, records=1, input=forward.1
[2024/05/28 06:49:58] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:49:58] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:49:58] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:49:58] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:49:58] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:49:58] [debug] [out flush] cb_destroy coro_id=23
[2024/05/28 06:49:58] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:50:00] [debug] [input chunk] update output instances with new chunk size diff=14699, records=1, input=forward.1
[2024/05/28 06:50:00] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:50:00] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:50:00] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:50:00] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:50:00] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:50:00] [debug] [out flush] cb_destroy coro_id=24
[2024/05/28 06:50:00] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:50:02] [debug] [input chunk] update output instances with new chunk size diff=924, records=1, input=forward.1
[2024/05/28 06:50:02] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:50:02] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:50:02] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:50:02] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:50:02] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:50:02] [debug] [out flush] cb_destroy coro_id=25
[2024/05/28 06:50:02] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:50:02] [debug] [input chunk] update output instances with new chunk size diff=14699, records=1, input=forward.1
[2024/05/28 06:50:03] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:50:03] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:50:03] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:50:03] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:50:03] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:50:03] [debug] [out flush] cb_destroy coro_id=26
[2024/05/28 06:50:03] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:50:03] [debug] [input chunk] update output instances with new chunk size diff=14699, records=1, input=forward.1
[2024/05/28 06:50:04] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:50:04] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:50:04] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:50:04] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:50:04] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:50:04] [debug] [out flush] cb_destroy coro_id=27
[2024/05/28 06:50:04] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:50:04] [debug] [input chunk] update output instances with new chunk size diff=14699, records=1, input=forward.1
[2024/05/28 06:50:05] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:50:05] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 has been assigned (recycled)
[2024/05/28 06:50:05] [debug] [http_client] not using http_proxy for header
[2024/05/28 06:50:05] [debug] [output:loki:loki.1] 192.168.133.167:3100, HTTP status=204
[2024/05/28 06:50:05] [debug] [upstream] KA connection #53 to 192.168.133.167:3100 is now available
[2024/05/28 06:50:05] [debug] [out flush] cb_destroy coro_id=28
[2024/05/28 06:50:05] [debug] [task] destroy task=0x7f88de244b60 (task_id=0)
[2024/05/28 06:50:06] [debug] [input chunk] update output instances with new chunk size diff=14699, records=1, input=forward.1
[2024/05/28 06:50:06] [debug] [task] created task=0x7f88de244b60 id=0 OK
[2024/05/28 06:50:06] [engine] caught signal (SIGSEGV)
AWS for Fluent Bit Container Image Version 2.32.2.20240425
June3Ningxu commented 2 weeks ago

will use the tag of "debug-2.32.2.20240516" and test again