newrelic / newrelic-ruby-agent

New Relic RPM Ruby Agent
https://docs.newrelic.com/docs/apm/agents/ruby-agent/getting-started/introduction-new-relic-ruby/
Apache License 2.0
1.2k stars 597 forks source link

Add ability to sort by individual Redis instances #2290

Open ohinton opened 10 months ago

ohinton commented 10 months ago

Is your feature request related to a problem? Please describe.

I have a customer reporting that they use 2 separate Redis databases, and they want to sort by individual Redis instances on several different views (the "Databases" view, "Web transaction time" chart), so they can tell which slow operations are from which instance.

Feature Description

Add the ability to see all Redis instances separated. The Redis instance name could be based on the Redis host name. For example, "Redis GET" operation is executed against"Redis host1" or "Redis host2" instance.

Additional context

Heroku case: https://heroku.support/1297031

Priority

Really Want

Internal Resources

workato-integration[bot] commented 10 months ago

https://new-relic.atlassian.net/browse/NR-173260

kaylareopelle commented 10 months ago

Internal - Slack Discussion

kaylareopelle commented 10 months ago

Update November 3, 2023

This sounds like a really awesome feature that would hold value for many customers. Thanks for your request! We've done a bit of discovery and, unfortunately, the effort to make this happen is pretty high.

The metrics that drive the content on the "Databases" page and the "Web transaction time" chart aren't easily updated. In addition to making changes to the Ruby agent, we'd also need changes from multiple UI teams to make this happen.

We'll keep the discussion on this idea open and will update this issue with any new developments regarding when we might be able to take this on.

If anyone else comes across this issue and would also like to see it implemented, please react with an emoji on the issue description.

While we're waiting, an alternative approach to get similar data would be to use Span events to generate similar charts on a custom dashboard.

One query you could experiment with is:

SELECT average(duration) 
FROM Span 
WHERE entity.guid = '<YOUR_ENTITY>' 
AND name LIKE 'Datastore/operation/Redis/get' 
FACET peer.hostname 
TIMESERIES

Visit the New Relic Docs site for more information on how to query your data.

ArturT commented 10 months ago

@kaylareopelle Thanks for the update.

If you have distributed tracing enabled then the other workaround to identify slow Redis operations coming from different Redis hosts is to open APM & Services -> (choose your app) -> Transactions -> (select something from Top 20 transactions, for example select a background sidekiq worker) -> on the Transaction traces table select one of slow transactions -> select the Database queries tab -> In the Database instance column you can see host name (i.e. Redis host1 or host2).

But if you selected API endpoint instead of background sidekiq worker then Redis host name is not visible. It's just showing web.1 for Heroku dyno name. Not sure why it's showing correctly Redis host name only for background sidekiq workers.

fallwith commented 10 months ago

Hi @ArturT,

Thank you for sharing the workaround.

But if you selected API endpoint instead of background sidekiq worker then Redis host name is not visible. It's just showing web.1 for Heroku dyno name. Not sure why it's showing correctly Redis host name only for background sidekiq workers.

For this part, does the web.1 dyno perform DT traced requests to the Redis dyno? If there's either a bug or a feature request related to how the Redis dyno name gets surfaced to New Relic for API requests, we'd love to know more about it and surface it as a separate GitHub issue. Especially given the complexity that @kaylareopelle described with this feature request, we'd like to see if anything else can be done to help multi Redis host users.

ArturT commented 10 months ago

For this part, does the web.1 dyno perform DT traced requests to the Redis dyno?

I believe yes, because we use default NewRelic settings (DT tracing is enabled).

To keep this issue separate I created a new issue:

fallwith commented 10 months ago

To keep this issue separate I created a new issue

Perfect. Thanks, @ArturT!