fluent / fluent-bit

Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX and Windows
https://fluentbit.io
Apache License 2.0
5.77k stars 1.57k forks source link

time key is missing from the sent data when using the splunk output with Splunk_Send_Raw on #8926

Open pmeier opened 3 months ago

pmeier commented 3 months ago

Bug Report

Describe the bug

time key is missing from the sent data when using the splunk output with Splunk_Send_Raw On

To Reproduce

  1. Spin up a Splunk mockup server on port 8088 that just prints the data it gets sent.

    Example

    Python / uvicorn / FastAPI ```python from fastapi import FastAPI, Request import json app = FastAPI() @app.post("/{path:path}") async def mirror(request: Request, path: str) -> None: body = await request.body() line = body.splitlines()[0] record = None for i in range(len(line), 1, -1): try: record = json.loads(line[:i]) except Exception: pass if record is None: out = body else: out = json.dumps(record, indent=2) print(out) print("-" * 80) ```

  2. Run the example for Splunk_Send_Raw on from the documentation.

    [INPUT]
        Name  cpu
        Tag   cpu
    
    # nest the record under the 'event' key
    [FILTER]
        Name nest
        Match *
        Operation nest
        Wildcard *
        Nest_under event
    
    # add event metadata
    [FILTER]
        Name      modify
        Match     *
        Add index my-splunk-index
        Add host  my-host
    
    [OUTPUT]
        Name        splunk
        Match       *
        Host        127.0.0.1
        Splunk_Token xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx
        Splunk_Send_Raw On
  3. Observe that there is no time entry in the output of the mockup server

    {
      "event": {
        "cpu_p": 1.083333333333333,
        "user_p": 0.875,
        "system_p": 0.2083333333333333,
        ...  # omitted for brevity
      },
      "index": "my-splunk-index",
      "host": "my-host"
    }
    --------------------------------------------------------------------------------
    INFO:     127.0.0.1:42188 - "POST /services/collector/raw HTTP/1.1" 200 OK

Expected behavior

Have a time key in the record as stated in the documentation:

{
    "time": "1535995058.003385189",
    "index": "my-splunk-index",
    "host": "my-host",
    "event": {
        "cpu_p":0.000000,
        "user_p":0.000000,
        "system_p":0.000000
    }
}

Your Environment

Additional context

The time entry is added correctly when setting Splunk_Send_Raw Off. Re-running the example from above gives the following output:

{
  "time": 1717750122.124004,
  "event": {
    "event": {
      "cpu_p": 1.916666666666667,
      "user_p": 1.625,
      "system_p": 0.2916666666666667,
      ...  # omitted for brevity
    },
    "index": "my-splunk-index",
    "host": "my-host"
  }
}
github-actions[bot] commented 3 weeks ago

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

pmeier commented 3 weeks ago

This is still broken.