search

dynatrace-oss / logstash-output-dynatrace

A Logstash output plugin for sending logs to Dynatrace
4 stars 1 forks source link

Intermittent 400 Errors #38

Open HMarvellNW opened 3 weeks ago

HMarvellNW commented 3 weeks ago

Describe the bug Hello,

I'm currently running into some issues intermittently with the Logstash Output Plugin. Our current solution is passing the data through an Elastic Logstash instance with no filtering, where we can see that some logs are producing 400 errors, most logs are making it through to Dynatrace but some are running into this issue. We have been able to confirm within our Elastic solution that this issue is being caused by a 400 Error. Due to this issue being intermittent we are confident that the tenant ID and Dynatrace tokens are all valid and working as expected.

To Reproduce Steps to reproduce the behavior: We are currently using an output of the following within out Logstash Pipeline. As mentioned the Tenant_UUID and DT_TOKEN variables are both valid and working.

`output {

dynatrace {

 id => "dynatrace_output"

 ingest_endpoint_url => "[https://127.0.0.1:9999/e/${TENANT_UUID}/api/v2/logs/ingest"]

 api_key => "${DT_TOKEN}"

 ssl_verify_none => true

}

}`

Expected behavior I would expect there to be no 400 errors unless there was an ongoing issue.

Screenshots Screenshot 2024-10-11 at 14 48 47

Additional context It looks like the issue has subsided recently, but due to the nature of this being an ongoing issue I would still like to raise this with yourselves.

dyladan commented 3 weeks ago

Hello. 400 errors are most often caused by invalid JSON or by exceeding some API limit. Can you please include what version of the output plugin is installed? If possible please include any relevant debug logs from your logstash instance.