search

newrelic / newrelic-java-agent

The New Relic Java agent
Apache License 2.0
199 stars 143 forks source link

Prevent the Agent from storing raw SQL queries in memory when "record_sql" is "off" or "obfuscated" #461

Open kford-newrelic opened 3 years ago

kford-newrelic commented 3 years ago

It looks like the Agent will currently store any full, raw SQL query that it detects until the end of the Tracer. At that point it checks the Agent's "record_sql" configuration and either obfuscates the query or drops it entirely. Unfortunately we've seen a few instances where customers have a huge number of large SQL queries being executed which causes the Agent to take up massive amounts of heap space over short periods.

Since the "record_sql" configuration doesn't prevent the raw queries from being stored in memory we don't have any good options of resolving the issue except by disabling the database instrumentation entirely. Allowing the "record_sql" configuration to immediately affect the data that gets stored in memory, either by obfuscating that data first or dropping it entirely(based on the exact configuration), would give Support a path forward to prevent raw SQL query data from using memory space, if needed.

(Migrate to Jira)

kford-newrelic commented 2 years ago

Comment added by Kevyn Ford in Aha! - View

Snapshot agent build provided to customer for feedback. Moving this MMF to Nov to compensate for the feedback delay.

kford-newrelic commented 2 years ago

Current approach in #559 has concerns, including breaking an underlying public API.

Going to have to re-prioritize this for a future round of engineering work.

kford-newrelic commented 2 years ago

Comment added by Kevyn Ford in Aha! - View

Could not make the 7.5.0 release due to feature conflict concerns. Will review for a potential future release.

kford-newrelic commented 2 years ago

Will likely need another ~2 weeks of engineering time

workato-integration[bot] commented 2 years ago

https://issues.newrelic.com/browse/NEWRELIC-4065