GoogleCloudPlatform / ops-agent

Apache License 2.0
138 stars 66 forks source link

Ops Agent fails intermittently to execute type modify_fields instruction #1699

Open PawelKGalka opened 2 months ago

PawelKGalka commented 2 months ago

Describe the bug Havin exemplary field in jsonPayload.auditLog.rawData.txTimeMillis it is not always parsed to integer in Logs Explorer and remains as string

To Reproduce Steps to reproduce the behavior:

  1. Start a GCE VM with image RHEL 8
  2. Install Ops Agent version 2.46.1
  3. Customize the agent with configuration
    logging:
    receivers:
    messages:
      type: files
      include_paths:
      - /var/log/messages
    secure:
      type: files
      include_paths:
      - /var/log/secure
    applogs_tcp:
      type: tcp
      format: json
      listen_port: 5160
    processors:
    parse:
      type: parse_json
      time_key: jsonPayload.timestamp
      time_format: "%Y-%m-%dT%H:%M:%S.%L%Z"
    adjust_fields_to_stackdriver:
      type: modify_fields
      fields:
        jsonPayload.auditLog.rawData.txTimeMillis:
          type: integer
    service:
    pipelines:
      default_pipeline:
        receivers: [messages, secure]
      applogs_pipeline:
        receivers: [applogs_tcp]
        processors: [parse, adjust_fields_to_stackdriver]

    and restart the agent

  4. See some values are still in String format image

Expected behavior Value is in integer format always

Environment (please complete the following information):

Additional context NA

jefferbrecht commented 1 month ago

Hi @PawelKGalka, we recently released Ops Agent 2.47.0 which upgrades the version of Fluent Bit used internally. Are you able to upgrade to the latest version and reproduce the issue still?

If you're still able to reproduce this, can you provide any sample input logs that you're sending to that TCP port so that we can try to reproduce this ourselves? Feel free to open a Cloud Support case as well which would let us troubleshoot much more closely (you can reference this GitHub issue in that case if you'd like).