elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.52k stars 8.06k forks source link

[Fleet] Cannot add integration policy - handlebar template evaluation problem #181495

Open andrewkroh opened 3 months ago

andrewkroh commented 3 months ago

Fleet fails to create an integration package policy for aws_logs when using the parsers configuration setting.

Screenshot 2024-04-23 at 16 42 38

Kibana version: 8.13.2

Steps to reproduce:

  1. Add integration policy to a Fleet agent policy.
PUT kbn:/api/fleet/package_policies/3a08dc8c-f304-4bc6-bf10-157b170f8353
{
  "package": {
    "name": "aws_logs",
    "version": "1.1.0"
  },
  "name": "aws_logs-1",
  "namespace": "",
  "description": "",
  "policy_id": "0f10626f-11ff-40ea-a579-fcdbb5825654",
  "vars": {
    "endpoint": "",
    "default_region": ""
  },
  "inputs": {
    "aws_logs-aws-s3": {
      "enabled": true,
      "streams": {
        "aws_logs.generic": {
          "enabled": true,
          "vars": {
            "number_of_workers": 1,
            "bucket_list_interval": "120s",
            "file_selectors": [],
            "fips_enabled": false,
            "include_s3_metadata": [],
            "max_bytes": "10MiB",
            "max_number_of_messages": 5,
            "parsers": "- multiline:\n    pattern: '\\s*{$'\n",
            "queue_url": "http://foo.example.com",
            "sqs.max_receive_count": 5,
            "sqs.wait_time": "20s",
            "tags": [
              "forwarded"
            ],
            "preserve_original_event": false,
            "data_stream.dataset": "aws_logs.generic",
            "custom": ""
          }
        }
      }
    }
  }
}

The problematic part is the parsers configuration which was entered into the UI as

- multiline:
    pattern: '\s*{$'
  1. Submit the policy and an error is returned.
{"statusCode":500,"error":"Internal Server Error","message":"unexpected end of the stream within a single quoted scalar at line 25, column 1:\n    \n    ^"}

This is another example parser config that fails.

- multiline:
    type: pattern
    pattern: '\s*{$'
    flush_pattern: '^\s*}$'
    negate: true
    match: after

Related

elasticmachine commented 3 months ago

Pinging @elastic/fleet (Team:Fleet)