Allow for conversations and intent classifications to be matched to a particular model

[koernerfelicia]

Description of Problem: Currently, there is no way to to attribute a conversation or intent classification of an utterance in Rasa X to a particular model aside from comparing timestamps, and even this may not be foolproof. The ability to match a conversation is necessary for evaluation or comparisons of models (for example for A/B testing).

Consider a set of conversations downloaded from Rasa X for evaluation. Figuring out which of these conversations were produced by the target model is cumbersome, and at present could only be done by looking at which models were trained when and matching these to the timestamps in the conversation.

Overview of the Solution: Model information could be attached to events from Rasa OSS, or to the tracker. For evaluation of policies, it may be sufficient to have this information on a conversation level, whereas for NLU, it may be better to have this information per utterance.

Definition of Done:

• [x] Tests are added
• [x] Feature described in the docs -> not sure if this needs to be in the docs, though it should be in the specs.
• [x] Feature mentioned in the changelog
• [ ] Conversations in Rasa X can be matched to a particular model (fingerprint), and intent classifications of utterances can be matched to a particular model fingerprint

[TyDunn]

Exalate commented:

TyDunn commented:

@koernerfelicia @wochinge can this be closed now that https://github.com/RasaHQ/rasa/issues/8914 is closed?

[wochinge]

Exalate commented:

wochinge commented:

All points of the Definition of Done except the last one are done

[sync-by-unito[bot]]

➤ Maxime Verger commented:

:bulb: Heads up! We're moving issues to Jira: https://rasa-open-source.atlassian.net/browse/OSS.

From now on, this Jira board is the place where you can browse (without an account) and create issues (you'll need a free Jira account for that). This GitHub issue has already been migrated to Jira and will be closed on January 9th, 2023. Do not forget to subscribe to the corresponding Jira issue!

:arrow_right: More information in the forum: https://forum.rasa.com/t/migration-of-rasa-oss-issues-to-jira/56569.