Demonstrate one possible assistant flow targeted at new Studio developers with some C# experience.
Concept:
Create new training data set by converting existing C# code in public Altinn 3 app repositories into a repository of validation logic defined in structured English. This allows for a standard RAG approach to querying for relevance while limiting to a very specific domain (ValidateData, ValidateTask, ProcessNext, etc)
Shortcuts:
For demo purposes, we propose limiting the scope to a single domain, ValidateData. Note, many simple validation rules are handled by field-level configuration and not in C#.
User interface:
Interaction can be via Slack chat bot, VS Code plugin, directly in Studio, etc. For this demo, we will extend the existing Studio Assistant demo with a new keyword [validatedata]
Potential:
If the concept works well, it should be possible to generalize to multiple parallel domains, with either an agent-based categorizer.
### Training
- [x] Ingest InstanceValidator.cs files, parsing to include only ValidateData or ValidateTask methods (and dependencies)
- [x] Generate English-language version of validation rule from C# implementation, in structured format
- [x] Generate embeddings from rule summaries
### Inference
- [x] Test inference quality with manual prompts
- [x] Use vector database to find similar documents
- [x] Use document metadata to locate original source file
- [x] Format retrieved code
- [ ] Replace manual prompts with real user input (field name, rule description, error message)
- [ ] Deploy Studio Assistant slack bot to suitable hosting environment for broader testing
- [ ] Test results with a broader audience, identifying quality issues
- [ ] Use Azure OpenAI endpoints instead of OpenAI
Description
Demo objective:
Demonstrate one possible assistant flow targeted at new Studio developers with some C# experience.
Concept:
Create new training data set by converting existing C# code in public Altinn 3 app repositories into a repository of validation logic defined in structured English. This allows for a standard RAG approach to querying for relevance while limiting to a very specific domain (ValidateData, ValidateTask, ProcessNext, etc)
Shortcuts:
For demo purposes, we propose limiting the scope to a single domain, ValidateData. Note, many simple validation rules are handled by field-level configuration and not in C#.
User interface:
Interaction can be via Slack chat bot, VS Code plugin, directly in Studio, etc. For this demo, we will extend the existing Studio Assistant demo with a new keyword [validatedata]
Potential:
If the concept works well, it should be possible to generalize to multiple parallel domains, with either an agent-based categorizer.