search

codeforequity-at / botium-core

The Selenium for Chatbots - Bots Testing Bots
https://www.botium.ai
MIT License
229 stars 64 forks source link

Directline3 connection issue #278

Closed vijaysimh closed 5 years ago

vijaysimh commented 5 years ago

Hi am using directline3@0.0.12 with botium-cli version 0.0.48 to work on utterances.

Running command: botium-cli run mochawesome --config=./spec/botium.json --convos=./spec/convo --verbose

Below is the error captured in Console :In this why am getting error as 'unsubscribing from directline activity subscription +1ms'. What is the solution for connecting Directline3 and how to increase timeout.

C:\Users\Vijay\workspace\Portal>botium-cli run mochawesome --config=./spec/botium.json --convos=./spec/convo --verbose botium-cli Using Botium configuration file ./spec/botium.json +0ms botium-cli-run command options: { _: [ 'run' ], botium-cli-run help: false, botium-cli-run h: false, botium-cli-run version: false, botium-cli-run V: false, botium-cli-run config: './spec/botium.json', botium-cli-run c: './spec/botium.json', botium-cli-run convos: [ './spec/convo' ], botium-cli-run C: [ './spec/convo' ], botium-cli-run verbose: true, botium-cli-run v: true, botium-cli-run output: 'mochawesome', botium-cli-run testsuitename: 'Botium Test-Suite', botium-cli-run n: 'Botium Test-Suite', botium-cli-run expandutterances: false, botium-cli-run expandscriptingmemory: false, botium-cli-run timeout: 60, botium-cli-run '$0': botium-cli-run '..\..\AppData\Roaming\npm\node_modules\botium-cli\bin\botium-cli.js', botium-cli-run filter: undefined, botium-cli-run 'reporter-options': undefined } +0ms

2) Utter_5-A2/UTTER_NO._5-L5 botium-connector-directline3 Stop called +1m botium-connector-directline3 unsubscribing from directline activity subscription +1ms botium-connector-directline3 unsubscribing from directline connectionstatus subscription +0ms botium-connector-directline3 Start called +1ms botium-connector-directline3 Directline Connection Status: 1 / Connecting +1ms botium-cli-run running testcase Utter_5-A2/UTTER_NO._5-L6 +1m botium-Convo Utter_5-A2/UTTER_NO._5-L6/Cell A2: user says { botium-Convo "sender": "me", botium-Convo "messageText": "Which is the nearest place for getting detail?", botium-Convo "stepTag": "Cell A2", botium-Convo "not": false, botium-Convo "asserters": [], botium-Convo "logicHooks": [], botium-Convo "userInputs": [] botium-Convo } +1m botium-connector-directline3 UserSays called +2ms botium-connector-directline3 Posting activity { "from": { "id": "me" }, "type": "message", "text": "Which is the place for getting details?" } +1ms

10) Utter_5-A2/UTTER_NO._5-L9 botium-connector-directline3 Stop called +60s botium-connector-directline3 unsubscribing from directline activity subscription +2ms botium-connector-directline3 unsubscribing from directline connectionstatus subscription +5ms botium-connector-directline3 Clean called +3ms botium-BaseContainer Cleanup rimrafing temp dir C:\Users\Vijay\workspace\Portal\botiumwork\BotTesting-20190505-080144-h0sia +0ms

0 passing (16m) 16 failing

1) Botium Test-Suite Utter_5-A2/UTTER_NO._5-L1: Error: Timeout of 60000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

codeforequity-at commented 5 years ago

Those log messages are not alarming, it is just a notification that the Directline connection is built, subscribed and closed for each individual test case.

More interesting is the output "Directline Connection Status: 1 / Connecting", because it means that the Directline connection is not online. Increasing timeouts doesn't help in this case, you should make sure that your Chatbot is accessible over Directline.

vijaysimh commented 5 years ago

Sure thanks for the feedback and i shall check with the developers for the Directline Connection is Online and how to verify that Chatbot is accessible over directLine3. Please suggest the steps to be taken.

Below is the configuration for botium.json

{ "botium": { "Capabilities": { "PROJECTNAME": "VATesting", "CONTAINERMODE": "directline3", "CLEANUPTEMPDIR": true, "DIRECTLINE3_SECRET": "XXXXX", "DIRECTLINE3_WEBSOCKET": true, "DIRECTLINE3_POLLINGINTERVAL": 1000, "SCRIPTING_NORMALIZE_TEXT": true, "SCRIPTING_FORMAT": "xlsx", "SCRIPTING_XLSX_STARTROW": 2, "SCRIPTING_XLSX_STARTCOL": 1, "SCRIPTING_XLSX_SHEETNAMES":"Utter_5", "SCRIPTING_XLSX_SHEETNAMES_UTTERANCES":"Utterances" }, "Sources": {}, "Envs": { "NODE_TLS_REJECT_UNAUTHORIZED": 0 } } }

vijaysimh commented 5 years ago

It is authenticated with username and password , here developers are generating API Token which get expires once the session is ended and new token gets generated . Please suggest how to debug further and in botium.json how to authenticate.

codeforequity-at commented 5 years ago

Generating a Token for each session is currently not supported by our Directline connector. you will have to use the Directline Secret for connecting.

vijaysimh commented 5 years ago

After workaround came to know the company policy which is blocking the DirectLine connection which has to enabled in proxy server, but to whitelist it i need to know what are the port and IpAddress to be whiteListed in company server which Botium Listening :

Now am getting the following error: 1) Botium Test-Suite HelloUtt-A2/HELLO_UTT-L1: TranscriptError: Error: HelloUtt-A2/HELLO_UTT-L1/Cell B3: bot says nothing at async.waterfall (C:\Users\VIjay\AppData\Roaming\npm\node_modules\botium-cli\node_modules\botium-core\src\scripting\Convo.js:214:25) at C:\Users\Vijay\AppData\Roaming\npm\node_modules\botium-cli\node_modules\async\dist\async.js:473:16 at next (C:\Users\Vijay\AppData\Roaming\npm\node_modules\botium-cli\node_modules\async\dist\async.js:5329:29) at C:\Users\Vijay\AppData\Roaming\npm\node_modules\botium-cli\node_modules\async\dist\async.js:969:16 at runConversation (C:\Users\Vijay\AppData\Roaming\npm\node_modules\botium-cli\node_modules\botium-core\src\scripting\Convo.js:192:15) at async.mapSeries (C:\Users\Vijay\AppData\Roaming\npm\node_modules\botium-cli\node_modules\botium-core\src\scripting\Convo.js:383:9) at C:\Users\Vijay\AppData\Roaming\npm\node_modules\botium-cli\node_modules\async\dist\async.js:1140:9 at C:\Users\Vijay\AppData\Roaming\npm\node_modules\botium-cli\node_modules\async\dist\async.js:473:16 at iterateeCallback (C:\Users\Vijay\AppData\Roaming\npm\node_modules\botium-cli\node_modules\async\dist\async.js:988:17) at C:\Users\Vijay\AppData\Roaming\npm\node_modules\botium-cli\node_modules\async\dist\async.js:969:16

codeforequity-at commented 5 years ago

Botium and Directline is not listening. It just opens outbound connections.

vijaysimh commented 5 years ago

DirectLine3 using Excel what is the approach to hit on multiple Buttons with the same response - as per the below attached screenshot of UI.

image

Please find the Error occured: TranscriptError: Error: HelloUtt-A8/BOT_GREETING_UTT-L1/Cell B10: bot says nothing

Please find the excel screenshot what i have tried and please let me know the correct procedure to click on Button with Decision making. image

codeforequity-at commented 5 years ago

What your Excel does is to tell Botium to verify that the two buttons are available in the chatbot response. https://botium.atlassian.net/wiki/spaces/BOTIUM/pages/17399914/Buttons+Asserter

To simulate a button click, you can use the BUTTON user input method:

#me
BUTTON Delete PO

see https://botium.atlassian.net/wiki/spaces/BOTIUM/pages/491664/Botium+Scripting+-+BotiumScript

But the "bot says nothing" error means something different, the chatbot answer is not received by Botium, maybe again connectivity issues with Directline.

vijaysimh commented 5 years ago

On workAround had installed Botium in Virtual machine which has Zscalar a firewall tool,if zscalar enabled its blocking the execution of Convos - as am getting Error as "timeout 60000ms" and connection fails and after we disable zscalar firewall convos(Utterances and Responses) are validated correctly with connection established and passed. What can be the reason behind the proxy which is blocking the botium-direcline3-connector .Please let us know as i have to raise for IT team what's blocking the way. Following steps have tried to overcome: 1.Whether i need to provide port Listening/Http URL of directLine3 or secret key directLine -botium.json in firewall to exclude . 2.Am trying to check whether the folder "nodule_modules/botframework-directlinejs" if its added in firewall to exclude ,whether its connects with no hurdle.

codeforequity-at commented 5 years ago

Directline connects to the Azure Bot Service on this URL: https://directline.botframework.com/v3/directline You should configure your firewall to allow connection to this URL for node.js processes.

vijaysimh commented 5 years ago

Thanks for the guideLines as suggested shall add exception to domain name "https://directline.botframework.com/v3/directline" and shall post regarding the status.

vijaysimh commented 5 years ago

Thank you resolved ..