codeforequity-at / botium-core

The Selenium for Chatbots - Bots Testing Bots
https://www.botium.ai
MIT License
229 stars 64 forks source link

Allow for variables in me requests and bot responses #391

Closed troyibm closed 5 years ago

troyibm commented 5 years ago

Is your feature request related to a problem? Please describe. We have a need to run botium tests on different data sets. We can work around this by adding strings like {{PersonAName}} in .convo.txt files and then using sed script to replace the strings with an actual data value, but sure would be nice if we could use environment variables like ${BOTIUM_PersonAName} and botium CLI replace that environment variable in real time so we don't have to generate convo.txt files based on the data the bot is connected to.

Describe the solution you'd like In the botium.json file, could have a variables object where that variable would be replaced with the value in #me and #bot utterances when tests are run.

Describe alternatives you've considered Make our convo.txt files have the same variables, but have to use scripts to make copies of convo.txt files with strings replaced by sed. Have to add this scripts to our DevOps process.

Additional context

codeforequity-at commented 5 years ago

maybe this is what you are looking for ? https://botium.atlassian.net/wiki/spaces/BOTIUM/pages/48824412/Scripting+Memory+File

troyibm commented 5 years ago

Maybe. I have a question on the usage of this "Memory file" on page https://botium.atlassian.net/wiki/spaces/BOTIUM/pages/48824412/Scripting+Memory+File I'm not sure where the lines below "scripting memory for product" and "scripting memory for order number" are stored. If they are in the .convo.txt file then that won't work for us. If the memory file can be outside the convo.txt files, then I could have our script just modify (or generate) the memory file and leave all the convo.txt files with $variable names. That would be much less work for us.

To further explain our issue... let's say our order bot can order anything, so the bot is used by a grocery store and an electronics store, so the "scripting memory for product" needs to be different for each. Can't order bread at the electronics store.

codeforequity-at commented 5 years ago

You are using text files, so the scripting memory files are recognized with the .scriptingmemory.txt file extension, outside of the convo files.

troyibm commented 5 years ago

The examples only show the variables used in the #me requests. Can they be used in the #bot responses?

troyibm commented 5 years ago

The examples also show using the variable in the #me requests and not using the variable values, like we have a test like this...

#me
What can I order from you?

#bot
You can order Bread, Beer, and Eggs.

Can particular values from the .scriptingmemory.txt file be used like "$product1 $product2"?

And I assume the values for product1 and product2 can have multiple words and that's okay. For example:

            | $productName
product1|49 inch plasma television
product2|natural gas grill
codeforequity-at commented 5 years ago

So after giving some thoughts, my conclusion is that the scripting memory files are not a solution for your question. The scripting memory is more about capturing parts of the chatbot response for later reuse (example: bot response with an order number you want to use later in the conversation).

The good news is that your original suggestion, to use environment variable replacement, was just a very small code change and will be included in the next build (pull request #392). You can use environment variables in your convo file like this:

Add $func(process.env.BOTIUM_PRODUCT) to the shopping cart
troyibm commented 5 years ago

Cool. Sounds like that will work for #me requests and #bot response.

codeforequity-at commented 5 years ago

After giving some more thoughts, the scripting memory files could indeed be a good solution for this question as well.

You have a convo file representing the generic conversation part:

#me
What can I order from you?

#bot
You can order $available_products

#me
Ok, then send me some $productName

#bot
Added $productName to the shopping cart

Then you have a directory holding the scripting memory files:

                 | $available_products
product1  | Bread, Beer, Eggs
                 | $productName
product1  | Bread
product2  | Beer

And then use the botium-cli "--convos" flag to point to the directory holding the convo, and additionally to another directory holding the scripting memory files.

for this case, it is important to set the SCRIPTING_MEMORY_MATCHING_MODE capability to joker, as the values are not simple words but include spaces

I added exactly this example as unit test to the repository: https://github.com/codeforequity-at/botium-core/pull/392/commits/1add3f2db9f73717e8f24079457fa8988393c36e

codeforequity-at commented 5 years ago

news here ? can this issue be closed ?

troyibm commented 5 years ago

news here ? can this issue be closed ?

Thanks for asking. Yes, go ahead and close. I can reopen if we have issues. We aren't using this yet and likely will use the process.env.VAR first. It's easier to have a single file of env vars than to have a file for every list that we might provide in the skill (i.e. we might have many of those $available_products lists)

codeforequity-at commented 5 years ago

If you have a suggestion for a good solution when working with multiple lists, pls make another feature request. I currently don't have an idea what you want to reach.