splunkhecexporter field extraction truncates at 1000 characters

bdschaap commented 7 months ago

Component(s)

No response

What happened?

Description

Field extraction truncates at 1000 characters

Steps to Reproduce

Transmit a field with a value that is more than 1000 characters. I used a stack trace with a value that's well over 1000 characters

Expected Result

Field isn't truncated

Actual Result

Field is truncated at 1000 characters

Note that routing the log to the debug exporter will field the entire value of the field

Collector version

0.92

Environment information

Environment

OS: (e.g., "Ubuntu 20.04") Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

receivers:
  otlp:
    ...
exporters:
  splunk_hec:
    ...
  debug:
    verbosity: detailed
service:
  pipelines:
    logs:
      receivers: [otlp]
      exporters: [splunk_hec, debug]

Log output

No response

Additional context

I'm not certain where in the process field extraction and the truncation occurs. I'm receiving Otel logs and exporting them via the Splunk HEC exporter. Those logs are then sent to a Splunk Heavy Forwarder which then forwards it on to Splunk Cloud.

github-actions[bot] commented 7 months ago

Pinging code owners for exporter/splunkhec: @atoulme @dmitryax. See Adding Labels via Comments if you do not have permissions to add labels yourself.

github-actions[bot] commented 5 months ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

exporter/splunkhec: @atoulme @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.

bdschaap commented 5 months ago

This issue still exists. Have also been working w/ Splunk Support on troubleshooting.

atoulme commented 5 months ago

Can you see this issue if you build and send via curl a HEC event manually?

atoulme commented 3 months ago

Sorry, I still don't understand your issue here.

Can you provide a sample of input we can use to reproduce? What is a field here, is it a log attribute? A log body?

Can you try to send HEC to a HEC receiver on the same collector and out to debug? This will help us understand what we truncate.

atoulme commented 3 months ago

I suspect you have a regex that is misfiring past 1000 characters. See DEPTH_LIMIT here: https://docs.splunk.com/Documentation/SplunkCloud/latest/Knowledge/Configureadvancedextractionswithfieldtransforms

Please continue to work directly with Splunk support and let them know that I am available for troubleshooting. I am going to close this issue at this time.

open-telemetry / opentelemetry-collector-contrib