[Obs AI Assistant] Change Knowledge base category filters to use dedicated field

[miltonhultgren]

Summary

Today, we use labels.category to decide which type of Knowledge base entry we're dealing with. The issue with this is that the mapping for labels.category is set to dynamic so it is unpredictable, and changes based on which document is ingested first. Usually it ends up being a multi field with type test and a sub field with type keyword, all of which have impact on the results depending on which field and which query is being used.

labels are better suited for user driven categorization, and we would be better served for our internal use cases to have a dedicated field for category.

This would however require us to migrate existing labels.category into this new field that exist in the Knowledge base.

AC

• The mappings for labels.category is changed from dynamic: true to dynamic: runtime to ensure it's always type keyword
• A new mapping is added for category of type keyword
• The Knowledge base APIs are updated to accept and use this new field instead
• A migration step is added (likely to init) that changes the mappings of existing indices and looks for entries of the old format and rewrites them

[elasticmachine]

Pinging @elastic/obs-knowledge-team (Team:obs-knowledge)