Version 9.8.0 introduces instrumentation for ruby-openai, adds the option to store tracer state on the thread-level, hardens the browser agent insertion logic to better proactively anticipate errors, and prevents excpetions from being raised in the Active Support Broadcast logger instrumentation.
Feature: Add instrumentation for ruby-openai
Instrumentation has been added for the ruby-openai gem, supporting versions 3.4.0 and higher (PR#2442). While ruby-openai instrumentation is enabled by default, the configuration option ai_monitoring.enabled is disabled by default and controls all AI monitoring. ai_monitoring.enabled must be set to true in order to receive ruby-openai instrumentation. High-Security Mode must be disabled in order to receive AI monitoring.
Calls to embedding and chat completion endpoints are automatically traced. These events can be enhanced with the introduction of two new APIs. Custom attributes can also be added to LLM events using the API NewRelic::Agent.add_custom_attributes, but they must be prefixed with llm.. For example, NewRelic::Agent.add_custom_attributes({'llm.user_id': user_id}).
Feature: Add AI monitoring APIs
This version introduces two new APIs that allow users to record additional information on LLM events:
NewRelic::Agent.record_llm_feedback_event - Records user feedback events.
NewRelic::Agent.set_llm_token_count_callback - Sets a callback proc for calculating token_count attributes for embedding and chat completion message events.
Visit RubyDoc for more information on each of these APIs.
Feature: Store tracer state on thread-level
A new configuration option, thread_local_tracer_state, stores New Relic's tracer state on the thread-level, as opposed to the default fiber-level storage. This configuration is turned off by default. Our thanks go to community member @markiz who contributed the idea, code, configuration option, and tests for this new feature! PR#2475.
Bugfix: Harden the browser agent insertion logic
With Issue#2462, community member @miry explained that it was possible for an HTTP response headers hash to have symbols for values. Not only would these symbols prevent the inclusion of the New Relic browser agent tag in the response body, but more importantly they would cause an exception that would bubble up to the monitored web application itself. With PR#2465 symbol based values are now supported and all other potential future exceptions are now handled. Additionally, the refactor to support symbols has been shown through benchmarking to give the processing of string and mixed type hashes a slight speed boost too.
Bugfix: Prevent Exception in Active Support Broadcast logger instrumentation
Previously, in certain situations the agent could cause an exception to be raised when attempting to interact with a broadcast log event. This has been fixed. Thanks to @nathan-appere for reporting this issue and providing a fix! PR#2510
Commits
6976570 Merge pull request #2529 from newrelic/scorecard
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Bumps newrelic_rpm from 9.7.1 to 9.8.0.
Changelog
Sourced from newrelic_rpm's changelog.
Commits
6976570
Merge pull request #2529 from newrelic/scorecard8b3acc0
remove workflowbaeba47
Merge pull request #2526 from newrelic/prerelease_updates_9.8.0-pre92c7921
bump versionc9caa82
Merge pull request #2525 from newrelic/update_changelog_ruby-openai4591431
Update CHANGELOG summary to include ruby-openai instrumentation8f0b7e8
Merge pull request #2524 from newrelic/add_changelog_entry_for_community_PR2510716d5db
add to summaryf0585ab
add changelog entry7aef0a5
Merge pull request #2510 from emotion-studio/hotfix/nil-exceptionDependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase
.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show