Made use of drop-small-last-batch logic only possible in DIET and configurable

[twerkmeister]

Proposed changes:

• added config var for "drop small last batch" for DIET which is off by default and results in the original batching logic
• turned off the logic permanently for TED- and UnexpecTEDIntentPolicy
• added tests that batching logic reacts to the parameter
• did not add test for UnexpecTEDIntentPolicy not failing because it was way to complex to create the needed precomputation objects to really simulate a full training. In a naive test the policy did not fail at all. Reproduced the error locally as follows and confirmed that it is now gone

Error reproduction

• initialize a new bot repo using rasa init
• add UnexpecTEDIntentPolicy to the policies, with batch size 5
• remove rules
• replace stories.yml with the yml below containing 12 stories that will lead to two being dropped with batch size 5 and triggering the problem
• run rasa train --force with and without the changes

Stories

version: "3.1"

stories:

- story: happy path
  steps:
  - intent: greet
  - action: utter_greet
  - intent: mood_great
  - action: utter_happy

- story: sad path 1
  steps:
  - intent: greet
  - action: utter_greet
  - intent: mood_unhappy
  - action: utter_cheer_up
  - action: utter_did_that_help
  - intent: affirm
  - action: utter_happy

- story: sad path 2
  steps:
  - intent: greet
  - action: utter_greet
  - intent: mood_unhappy
  - action: utter_cheer_up
  - action: utter_did_that_help
  - intent: deny
  - action: utter_goodbye

- story: Say goodbye anytime the user says goodbye
  steps:
  - intent: goodbye
  - action: utter_goodbye

- story: Say 'I am a bot' anytime the user challenges
  steps:
  - intent: bot_challenge
  - action: utter_iamabot

- story: x
  steps:
  - intent: goodbye
  - action: utter_goodbye
  - intent: bot_challenge
  - action: utter_iamabot

- story: happy path w goodbye
  steps:
  - intent: greet
  - action: utter_greet
  - intent: mood_great
  - action: utter_happy
  - intent: goodbye 
  - action: utter_goodbye

- story: sad path w goodbye
  steps:
  - intent: greet
  - action: utter_greet
  - intent: mood_unhappy
  - action: utter_cheer_up
  - action: utter_did_that_help
  - intent: affirm
  - action: utter_happy
  - intent: goodbye
  - action: utter_goodbye

- story: sad path 2 w goodbye
  steps:
  - intent: greet
  - action: utter_greet
  - intent: mood_unhappy
  - action: utter_cheer_up
  - action: utter_did_that_help
  - intent: deny
  - action: utter_goodbye
  - intent: goodbye
  - action: utter_goodbye

- story: happy path w bot challenge
  steps:
  - intent: greet
  - action: utter_greet
  - intent: bot_challenge
  - action: utter_iamabot
  - intent: mood_great
  - action: utter_happy

- story: sad path 1 w bot challenge
  steps:
  - intent: greet
  - action: utter_greet
  - intent: bot_challenge
  - action: utter_iamabot
  - intent: mood_unhappy
  - action: utter_cheer_up
  - action: utter_did_that_help
  - intent: affirm
  - action: utter_happy

- story: sad path 2 w bot challenge
  steps:
  - intent: greet
  - action: utter_greet
  - intent: bot_challenge
  - action: utter_iamabot
  - intent: mood_unhappy
  - action: utter_cheer_up
  - action: utter_did_that_help
  - intent: deny
  - action: utter_goodbye

Status (please check what you already did):

• [x] added some tests for the functionality
• [ ] updated the documentation
• [x] updated the changelog (please check changelog for instructions)
• [x] reformat files using black (please check Readme for instructions)

[github-actions[bot]]

🚀 A preview of the docs have been deployed at the following URL: https://12948--rasahq-docs-rasa-v2.netlify.app/docs/rasa