RasaHQ / rasa

💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants
https://rasa.com/docs/rasa/
Apache License 2.0
18.61k stars 4.6k forks source link

[ATO-1315] Flows yaml schema #12901

Closed Urkem closed 11 months ago

Urkem commented 11 months ago

Proposed changes:

Status (please check what you already did):

Urkem commented 11 months ago

@ancalita The validation itself seems to fail on financial llm demo bot... (the flows got loaded. Still not sure if the flow name is a required field for a flow)

Screenshot 2023-10-10 at 13 26 59

The training works if I do skip validation (this does not skip checking the schema):

Screenshot 2023-10-10 at 13 24 54
ancalita commented 11 months ago

The validation itself seems to fail on financial llm demo bot

I think that's because pattern_code_change is missing a name, I think this should be part of the schema and should be mandatory, in which case schema validation should fail.

Urkem commented 11 months ago

I think that's because pattern_code_change is missing a name, I think this should be part of the schema and should be mandatory, in which case schema validation should fail.

Added name as required and pattern_code_change is a part of OSS so I updated it accordingly.

Urkem commented 11 months ago

@twerkmeister @ancalita running the e2e tests on the demo bot (with the default flow change and without the change) is 50pass 18fail always is that the expected ratio or do all tests pass?

Screenshot 2023-10-13 at 14 42 50

Rasa Version      :         3.8.0a11
Minimum Compatible Version: 3.5.0
Rasa SDK Version  :         3.8.0a1
Python Version    :         3.10.10
Operating System  :         macOS-13.6-arm64-arm-64bit
Python Path       :         /Users/urosmilovanovic/.pyenv/versions/3.10.10/envs/dm2-1-demo/bin/python
Rasa Plus Version  :         3.8.0a11
github-actions[bot] commented 11 months ago

🚀 A preview of the docs have been deployed at the following URL: https://12901--rasahq-docs-rasa-v2.netlify.app/docs/rasa

sonarcloud[bot] commented 11 months ago

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug C 5 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot E 1 Security Hotspot
Code Smell A 49 Code Smells

0.0% 0.0% Coverage
0.9% 0.9% Duplication

warning The version of Java (11.0.20) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17. Read more here

ancalita commented 11 months ago

@twerkmeister @ancalita running the e2e tests on the demo bot (with the default flow change and without the change) is 50pass 18fail always is that the expected ratio or do all tests pass?

Hey @Urkem Apologies, I missed this message on Friday, if the same tests fail without the schema change, I'd advise to open a ticket with the details about failing tests and alert Dan about it, should be investigated what change caused those tests to fail since the tests labelled as passing always passed (68 out of 68).