search

Arize-ai / open-inference-spec

A specification for OpenInference, a semantic mapping of ML inferences
https://arize-ai.github.io/open-inference-spec/
MIT License
44 stars 2 forks source link

Tools: should output from tools have a conventional key as well? #37

Open tslmy opened 8 months ago

tslmy commented 8 months ago

There is llm.output_messages for LLM calls. There is embedding.vector for embedding model calls. There is retrieval.documents for retrieval results. But there is no tool_call.output for tool calls:

https://github.com/Arize-ai/open-inference-spec/blob/d03ce2f508ff97e601478e96b2c0a22d9d0b5b11/trace/spec/semantic_conventions.md?plain=1#L42-L43

Should we add it?

I believe this is the cause of https://github.com/run-llama/llama_index/issues/9770 .

mikeldking commented 8 months ago

hi @tslmy thanks for filing an issue. Good call-out. the output of tools very often shows up in the subsequent LLM call but you are right - it's probably a good idea to capture this as part of the span. We will work with the LlamaIndex team to get visibility into this!