search

open-telemetry / semantic-conventions

Defines standards for generating consistent, accessible telemetry across a variety of domains
Apache License 2.0
274 stars 175 forks source link

GenAI: Clarify what `gen_ai.request.top_k` means #1397

Open lmolkova opened 2 months ago

lmolkova commented 2 months ago

OpenAI top_logprobs:

An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability. logprobs must be set to true if this parameter is used.

Cohere k:

Ensures only the top k most likely tokens are considered for generation at each step. Defaults to 0, min value of 0, max value of 500.

Anthropic top_k:

Only sample from the top K options for each subsequent token. Used to remove "long tail" low probability responses. Learn more technical details here.

We should document what this parameter is, currently it's not clear

https://github.com/open-telemetry/semantic-conventions/blob/26e3779ba8367c7f4c1d5c6e23000cdb51455d86/docs/gen-ai/gen-ai-spans.md?plain=1#L62