newrelic / nri-mssql

New Relic Infrastructure Mssql Integration
MIT License
8 stars 18 forks source link

Have text fields in mssql-custom-query.yml.sample queries truncated to NRDB attribute limit (4095) to avoid log bloat. #156

Open alaiavee opened 4 months ago

alaiavee commented 4 months ago

Description

In some environments the complete_text of SQL queries reported by some of the custom queries in mssql-custom-query.yml.sample is longer than the NRDB limit. This causes warning messages in the agent log about the value being truncated, including the truncated text. This can lead to considerable log bloat.

Example log: time="2024-04-18T10:09:31-04:00" level=warning msg="event truncated to NRDB limit" component=AgentContext entity_key="ms-instance:<REDACTED>:instance=<REDACTED>" length=129720 original="+map[displayName:<REDACTED> entityKey:ms-instance:EFP19-TR-SQL01:instance=efp19-tr-sql01 entityName:ms-instance:EFP19-TR-SQL01 eventType:MssqlCustomQuerySample event_type:MssqlCustomQuerySample host:EFP19-TR-SQL01 instance:<REDACTED> integrationName:com.newrelic.mssql integrationVersion:2.12.0 label.environment:Training label.product:<REDACTED> label.role:MSSQL longRunning_complete_text:CREATE PROCEDURE...<full text of query before truncation>

Acceptance Criteria

Update queries in mssql-custom-query.yml.sample so that text values do not exceed the NRDB attribute limit.

Describe Alternatives

Can work around the log bloat problem by either filtering out nri-mssql messages entirely per agent log config: https://github.com/newrelic/infrastructure-agent/blob/master/docs/log_configuration.md#log-filters

Can also work around by editing the sample query and forcing the complete_text to be truncated to the NRDB limit when queried: SELECT ... LEFT(t.[text], 4095) AS [complete_text] ...

Dependencies

Infrastructure Agent nri-mssql

NRDB limit checked by agent (section of the code that generates these log messages): https://github.com/newrelic/infrastructure-agent/blob/db359de4f1042f5fd6b20276629f6b37989fe265/internal/agent/agent.go#L1157

For Maintainers Only or Hero Triaging this bug

Suggested Priority (P1,P2,P3,P4,P5): Suggested T-Shirt size (S, M, L, XL, Unknown):

workato-integration[bot] commented 4 months ago

https://new-relic.atlassian.net/browse/NR-261812