jlewi
commented
2 months ago 196 added a unique log key "contextId" that groups all the traces related to a particular context.
The learning mechanism works even with streaming with a caveat.
Our logic for generating the traces is built around Agent.Generate. A streaming request will generate multiple traces but each one should produce a different block id so the actual block will be associated with its corresponding trace.
The big caveat is that for learning to occur a user has to
- Accept the ghost cell
- Edit and then execute the cell
If the user dismisses the ghost cell (e.g. inserts their own cell). No learning occurs.
We need to update tracing and learning to work with Autocomplete.
176 added tracing for streaming requests in a way that I think for a streaming request; all the generations would be assigned the same traceId. This will probably break our logic for producing and viewing traces because our code right now assumes each trace corresponds to a single LLM completion not N completions.
We'd like to be able to group all the completions related to a given cell/context. However, using a single traceId for the streaming request doesn't actually solve this because the streaming connection can timeout and be restarted.
It probably makes sense to come up with a suitable key for identifying when traces correspond to updating a particular completion context. We can then attach that join key to all the traces associated with it.