search

codeforequity-at / botium-core

The Selenium for Chatbots - Bots Testing Bots
https://www.botium.ai
MIT License
231 stars 64 forks source link

Getting Null response from Microsoft Directline3 chatbot #211

Closed Nimish121 closed 5 years ago

Nimish121 commented 5 years ago

botium-BotDriver GITDIR: '.' } +16ms botium-BotDriver Build - Envs : { IS_BOTIUM_CONTAINER: true, NODE_TLS_REJECT_UNAUTHORIZED: 0 } +2ms botium-PluginConnectorContainer Loading Botium plugin from directline3 failed - { Error: Cannot find module 'directline3' botium-PluginConnectorContainer at Function.Module._resolveFilename (internal/modules/cjs/loader.js:581:15) botium-PluginConnectorContainer at Function.Module._load (internal/modules/cjs/loader.js:507:25) botium-PluginConnectorContainer at Module.require (internal/modules/cjs/loader.js:637:17) botium-PluginConnectorContainer at require (internal/modules/cjs/helpers.js:22:18) botium-PluginConnectorContainer at tryLoadPlugin (D:\backup\sampleBotiumMocha\node_modules\botium-core\src\containers\PluginConnectorContainer.js:16:20) botium-PluginConnectorContainer at Validate.Validate.then (D:\backup\sampleBotiumMocha\node_modules\botium-core\src\containers\PluginConnectorContainer.js:45:29) code: 'MODULE_NOT_FOUND' } +0ms botium-PluginConnectorContainer Botium plugin botium-connector-directline3 loaded +183ms botium-connector-directline3 Validate called +0ms botium-connector-directline3 Build called +8ms botium-connector-directline3 Start called +9ms botium-connector-directline3 Directline Connection Status: 1 / Connecting +67ms botium-bindings-main running testcase1 testCase1.convo.txt +359ms botium-Convo testCase1.convo.txt/Line 3: user says { botium-Convo "sender": "me", botium-Convo "channel": null, botium-Convo "messageText": "insurance claim", botium-Convo "stepTag": "Line 3", botium-Convo "not": false, botium-Convo "asserters": [], botium-Convo "logicHooks": [] botium-Convo } +0ms botium-connector-directline3 UserSays called +33ms botium-connector-directline3 Directline Connection Status: 2 / Online +2s botium-connector-directline3 Posted activity, assigned ID G8hzXBlstqp8eBiXDXvkxC|0000000 +2s botium-Convo testCase1.convo.txt wait for bot null +4s botium-connector-directline3 received message { "type": "message", "id": "G8hzXBlstqp8eBiXDXvkxC|0000001", "timestamp": "2019-02-01T10:09:16.1817632Z", "localTimestamp": "2019-02-01T10:09:16.14+00:00", "channelId": "directline", "from": { "id": "Ginger_Bot", "name": "Ginger" }, "conversation": { "id": "G8hzXBlstqp8eBiXDXvkxC" }, "inputHint": "acceptingInput", "attachments": [ { "contentType": "application/vnd.microsoft.card.hero", "content": { "text": "Please contact Vishwanath Gambher to claim insurance.", "images": [ {} ], "tap": { "type": "openUrl", "title": "Insurance" } } } ], "replyToId": "G8hzXBlstqp8eBiXDXvkxC|0000000" } +7s botium-connector-directline3 received message { "type": "message", "id": "G8hzXBlstqp8eBiXDXvkxC|0000002", "timestamp": "2019-02-01T10:09:18.4390706Z", "localTimestamp": "2019-02-01T10:09:18.166+00:00", "channelId": "directline", "from": { "id": "Ginger_Bot", "name": "Ginger" }, "conversation": { "id": "G8hzXBlstqp8eBiXDXvkxC" }, "text": "Is this information helpful?", "inputHint": "expectingInput", "replyToId": "G8hzXBlstqp8eBiXDXvkxC|0000000" } +23ms botium-Convo testCase1.convo.txt: bot says { botium-Convo "sender": "bot", botium-Convo "sourceData": { botium-Convo "type": "message", botium-Convo "id": "G8hzXBlstqp8eBiXDXvkxC|0000001", botium-Convo "timestamp": "2019-02-01T10:09:16.1817632Z", botium-Convo "localTimestamp": "2019-02-01T10:09:16.14+00:00", botium-Convo "channelId": "directline", botium-Convo "from": { botium-Convo "id": "Ginger_Bot", botium-Convo "name": "Ginger" botium-Convo }, botium-Convo "conversation": { botium-Convo "id": "G8hzXBlstqp8eBiXDXvkxC" botium-Convo }, botium-Convo "inputHint": "acceptingInput", botium-Convo "attachments": [ botium-Convo { botium-Convo "contentType": "application/vnd.microsoft.card.hero", botium-Convo "content": { botium-Convo "text": "Please contact Vishwanath Gambher to claim insurance.", botium-Convo "images": [ botium-Convo {} botium-Convo ], botium-Convo "tap": { botium-Convo "type": "openUrl", botium-Convo "title": "Insurance" botium-Convo } botium-Convo } botium-Convo } botium-Convo ], botium-Convo "replyToId": "G8hzXBlstqp8eBiXDXvkxC|0000000" botium-Convo }, botium-Convo "media": [], botium-Convo "buttons": [], botium-Convo "cards": [ botium-Convo { botium-Convo "image": { botium-Convo "mimeType": "application/unknown" botium-Convo } botium-Convo } botium-Convo ], botium-Convo "messageText": null, botium-Convo "channel": "default" botium-Convo } +7s botium-ScriptingProvider assertBotResponse testCase1.convo.txt/Line 4 (Line 3: #me - insurance claim ) BOT: null = Please contact Vishwanath Gambher to claim insurance. ... +11s botium-bindings-main Test Case "testCase1.convo.txt" failed: { TranscriptError: Error: testCase1.convo.txt/Line 4: Expected bot response (on Line 3: #me - insurance claim ) "null" to match one of "Please contact Vishwanath Gambher to claim insurance." botium-bindings-main at async.waterfall (D:\backup\sampleBotiumMocha\node_modules\botium-core\src\scripting\Convo.js:160:25) botium-bindings-main at D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:473:16 botium-bindings-main at next (D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:5329:29) botium-bindings-main at D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:969:16 botium-bindings-main at runConversation (D:\backup\sampleBotiumMocha\node_modules\botium-core\src\scripting\Convo.js:144:15) botium-bindings-main at async.mapSeries (D:\backup\sampleBotiumMocha\node_modules\botium-core\src\scripting\Convo.js:317:9) botium-bindings-main at D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:1140:9 botium-bindings-main at D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:473:16 botium-bindings-main at iterateeCallback (D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:988:17) botium-bindings-main at D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:969:16 botium-bindings-main at D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:1137:13 botium-bindings-main at convoStepDone (D:\backup\sampleBotiumMocha\node_modules\botium-core\src\scripting\Convo.js:193:11) botium-bindings-main at scriptingEvents.onBotStart.then.then (D:\backup\sampleBotiumMocha\node_modules\botium-core\src\scripting\Convo.js:269:28) botium-bindings-main at process._tickCallback (internal/process/next_tick.js:68:7) botium-bindings-main name: 'TranscriptError', botium-bindings-main transcript: botium-bindings-main Transcript { botium-bindings-main steps: [ [TranscriptStep], [TranscriptStep] ], botium-bindings-main scriptingMemory: {}, botium-bindings-main convoBegin: 2019-02-01T10:09:09.915Z, botium-bindings-main convoEnd: 2019-02-01T10:09:21.011Z, botium-bindings-main err: botium-bindings-main Error: testCase1.convo.txt/Line 4: Expected bot response (on Line 3: #me - insurance claim ) "null" to match one of "Please contact Vishwanath Gambher to claim insurance." botium-bindings-main at ScriptingProvider.assertBotResponse (D:\backup\sampleBotiumMocha\node_modules\botium-core\src\scripting\ScriptingProvider.js:76:17) botium-bindings-main at scriptingEvents.onBotStart.then.then (D:\backup\sampleBotiumMocha\node_modules\botium-core\src\scripting\Convo.js:267:42) botium-bindings-main at process._tickCallback (internal/process/next_tick.js:68:7) } } +11s 1) testCase1.convo.txt botium-connector-directline3 Stop called +58ms botium-connector-directline3 unsubscribing from directline activity subscription +3ms botium-connector-directline3 unsubscribing from directline connectionstatus subscription +3ms botium-connector-directline3 Clean called +2ms

0 passing (11s) 1 failing

1) Botium Test Suite for samplebotiummocha testCase1.convo.txt: TranscriptError: Error: testCase1.convo.txt/Line 4: Expected bot response (on Line 3: #me - insurance claim ) "null" to match one of "Please contact Vishwanath Gambher to claim insurance." at async.waterfall (node_modules\botium-core\src\scripting\Convo.js:160:25) at D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:473:16 at next (node_modules\async\dist\async.js:5329:29) at D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:969:16 at runConversation (node_modules\botium-core\src\scripting\Convo.js:144:15) at async.mapSeries (node_modules\botium-core\src\scripting\Convo.js:317:9) at D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:1140:9 at D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:473:16 at iterateeCallback (node_modules\async\dist\async.js:988:17) at D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:969:16 at D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:1137:13 at convoStepDone (node_modules\botium-core\src\scripting\Convo.js:193:11) at scriptingEvents.onBotStart.then.then (node_modules\botium-core\src\scripting\Convo.js:269:28) at process._tickCallback (internal/process/next_tick.js:68:7)

Attacthng my botium.json, package.json and convo folder as well. New folder.zip

codeforequity-at commented 5 years ago

The bot doesn't return plain text, but it returns some multimedia content (application/vnd.microsoft.card.hero). I am currently adding some extensions to the directline connector, and I will work on this issue as well.

codeforequity-at commented 5 years ago

Please try the latest build.

Nimish121 commented 5 years ago

I tried after updating "botium-connector-directline3" but still getting same issue.

codeforequity-at commented 5 years ago

I used the project you attached above for testing it.

Nimish121 commented 5 years ago

I tried after updating the botium-connector-directline3 , below are the log details , please have a look

D:\backup\sampleBotiumMocha>npm install --save botium-connector-directline3 npm WARN samplebotiummocha@2.0.0 No description npm WARN samplebotiummocha@2.0.0 No repository field. npm WARN The package mocha is included as both a dev and production dependency. npm WARN The package mochawesome is included as both a dev and production dependency. npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.7 (node_modules\fsevents): npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.7: wanted {"os":"darwin","arch":"any"} (current: {"os":"win32","arch":"x64"})

D:\backup\sampleBotiumMocha>npm run test

samplebotiummocha@2.0.0 test D:\backup\sampleBotiumMocha mocha --reporter spec --exit spec

botium-bindings-main Botium Bindings args: { convodirs: [ './spec/convo' ], botium-bindings-main expandConvos: true, botium-bindings-main expandUtterancesToConvos: false } +0ms botium-BotDriver Loaded Botium configuration file ./botium.json +0ms botium-BotDriver BuildCompiler: Capabilites: { PROJECTNAME: 'samplebotiummocha', botium-BotDriver TEMPDIR: 'botiumwork', botium-BotDriver CLEANUPTEMPDIR: false, botium-BotDriver WAITFORBOTTIMEOUT: 10000, botium-BotDriver SIMULATE_WRITING_SPEED: false, botium-BotDriver DOCKERCOMPOSEPATH: 'docker-compose', botium-BotDriver DOCKERMACHINEPATH: 'docker-machine', botium-BotDriver DOCKERMACHINE: false, botium-BotDriver DOCKERIMAGE: 'node:boron', botium-BotDriver DOCKERUNIQUECONTAINERNAMES: false, botium-BotDriver DOCKERSYSLOGPORT_RANGE: '47100-47299', botium-BotDriver BOT_HEALTH_STATUS: 200, botium-BotDriver SLACK_PUBLISHPORT_RANGE: '46100-46299', botium-BotDriver FACEBOOK_PUBLISHPORT_RANGE: '46300-46499', botium-BotDriver FACEBOOK_SEND_DELIVERY_CONFIRMATION: true, botium-BotDriver BOTFRAMEWORK_PUBLISHPORT_RANGE: '46500-46699', botium-BotDriver BOTFRAMEWORK_WEBHOOK_PORT: 3978, botium-BotDriver BOTFRAMEWORK_WEBHOOK_PATH: 'api/messages', botium-BotDriver BOTFRAMEWORK_CHANNEL_ID: 'facebook', botium-BotDriver SIMPLEREST_METHOD: 'GET', botium-BotDriver WEBSPEECH_SERVER_PORT: 46050, botium-BotDriver WEBSPEECH_LANGUAGE: 'en-US', botium-BotDriver WEBSPEECH_CLOSEBROWSER: true, botium-BotDriver SCRIPTING_TXT_EOL: '\n', botium-BotDriver SCRIPTING_XLSX_EOL_SPLIT: '\r', botium-BotDriver SCRIPTING_XLSX_EOL_WRITE: '\r\n', botium-BotDriver SCRIPTING_XLSX_STARTROW: 1, botium-BotDriver SCRIPTING_XLSX_STARTCOL: 'A', botium-BotDriver SCRIPTING_NORMALIZE_TEXT: true, botium-BotDriver SCRIPTING_ENABLE_MEMORY: false, botium-BotDriver SCRIPTING_MATCHING_MODE: 'includeLowerCase', botium-BotDriver SCRIPTING_UTTEXPANSION_MODE: 'all', botium-BotDriver SCRIPTING_UTTEXPANSION_RANDOM_COUNT: 1, botium-BotDriver SCRIPTING_UTTEXPANSION_INCOMPREHENSION: 'INCOMPREHENSION', botium-BotDriver ASSERTERS: [], botium-BotDriver LOGIC_HOOKS: [], botium-BotDriver CONTAINERMODE: 'directline3', botium-BotDriver DIRECTLINE3_SECRET: botium-BotDriver 'z0vRUM70iEE.cwA.KOI.AI_pdzbo3iBASpMwUsW7l4N5gi2Mi_osZ8j83w6YkqQ', botium-BotDriver DIRECTLINE3_WEBSOCKET: true, botium-BotDriver DIRECTLINE3_POLLINGINTERVAL: 10000 } +4ms botium-ScriptingProvider Using matching mode: includeLowerCase +0ms botium-asserterUtils Loaded Default asserter - [ 'BUTTONS', 'MEDIA', 'PAUSE_ASSERTER' ] +0ms botium-asserterUtils Loaded Default logic hook - [ 'PAUSE', 'WAITFORBOT' ] +8ms botium-ScriptingProvider ReadConvosFromDirectory(./spec/convo) found filenames: botReply.utterances.txt,testCase1.convo.txt +18ms botium-ScriptingProvider ReadConvosFromDirectory(./spec/convo) found convos: botium-ScriptingProvider 1 testCase1.convo.txt ({ filename: 'testCase1.convo.txt' }): Line 3: #me - insurance claim | Line 4: #bot - Please contact Vishwanath Gambher to claim insurance. +5ms botium-ScriptingProvider ReadConvosFromDirectory(./spec/convo) found utterances: botium-ScriptingProvider botReply ({ filename: 'botReply.utterances.txt' }): Nice to know you care.|Thanks. I'm flattered.|I've been right here all along!|I didn't go anywhere +3ms botium-ScriptingProvider ExpandConvos - Using utterances expansion mode: all +12ms botium-bindings-main ready reading convos and utterances, number of test cases: (1). +332ms botium-bindings-main adding test case 1 testCase1.convo.txt +4ms

Botium Test Suite for samplebotiummocha botium-BotDriver Build - Capabilites: { PROJECTNAME: 'samplebotiummocha', botium-BotDriver TEMPDIR: 'botiumwork', botium-BotDriver CLEANUPTEMPDIR: false, botium-BotDriver WAITFORBOTTIMEOUT: 10000, botium-BotDriver SIMULATE_WRITING_SPEED: false, botium-BotDriver DOCKERCOMPOSEPATH: 'docker-compose', botium-BotDriver DOCKERMACHINEPATH: 'docker-machine', botium-BotDriver DOCKERMACHINE: false, botium-BotDriver DOCKERIMAGE: 'node:boron', botium-BotDriver DOCKERUNIQUECONTAINERNAMES: false, botium-BotDriver DOCKERSYSLOGPORT_RANGE: '47100-47299', botium-BotDriver BOT_HEALTH_STATUS: 200, botium-BotDriver SLACK_PUBLISHPORT_RANGE: '46100-46299', botium-BotDriver FACEBOOK_PUBLISHPORT_RANGE: '46300-46499', botium-BotDriver FACEBOOK_SEND_DELIVERY_CONFIRMATION: true, botium-BotDriver BOTFRAMEWORK_PUBLISHPORT_RANGE: '46500-46699', botium-BotDriver BOTFRAMEWORK_WEBHOOK_PORT: 3978, botium-BotDriver BOTFRAMEWORK_WEBHOOK_PATH: 'api/messages', botium-BotDriver BOTFRAMEWORK_CHANNEL_ID: 'facebook', botium-BotDriver SIMPLEREST_METHOD: 'GET', botium-BotDriver WEBSPEECH_SERVER_PORT: 46050, botium-BotDriver WEBSPEECH_LANGUAGE: 'en-US', botium-BotDriver WEBSPEECH_CLOSEBROWSER: true, botium-BotDriver SCRIPTING_TXT_EOL: '\n', botium-BotDriver SCRIPTING_XLSX_EOL_SPLIT: '\r', botium-BotDriver SCRIPTING_XLSX_EOL_WRITE: '\r\n', botium-BotDriver SCRIPTING_XLSX_STARTROW: 1, botium-BotDriver SCRIPTING_XLSX_STARTCOL: 'A', botium-BotDriver SCRIPTING_NORMALIZE_TEXT: true, botium-BotDriver SCRIPTING_ENABLE_MEMORY: false, botium-BotDriver SCRIPTING_MATCHING_MODE: 'includeLowerCase', botium-BotDriver SCRIPTING_UTTEXPANSION_MODE: 'all', botium-BotDriver SCRIPTING_UTTEXPANSION_RANDOM_COUNT: 1, botium-BotDriver SCRIPTING_UTTEXPANSION_INCOMPREHENSION: 'INCOMPREHENSION', botium-BotDriver ASSERTERS: [], botium-BotDriver LOGIC_HOOKS: [], botium-BotDriver CONTAINERMODE: 'directline3', botium-BotDriver DIRECTLINE3_SECRET: botium-BotDriver 'z0vRUM70iEE.cwA.KOI.AI_pdzbo3iBASpMwUsW7l4N5gi2Mi_osZ8j83w6YkqQ', botium-BotDriver DIRECTLINE3_WEBSOCKET: true, botium-BotDriver DIRECTLINE3_POLLINGINTERVAL: 10000 } +333ms botium-BotDriver Build - Sources : { LOCALPATH: '.', botium-BotDriver GITPATH: 'git', botium-BotDriver GITBRANCH: 'master', botium-BotDriver GITDIR: '.' } +11ms botium-BotDriver Build - Envs : { IS_BOTIUM_CONTAINER: true, NODE_TLS_REJECT_UNAUTHORIZED: 0 } +1ms botium-PluginConnectorContainer Loading Botium plugin from directline3 failed - { Error: Cannot find module 'directline3' botium-PluginConnectorContainer at Function.Module._resolveFilename (internal/modules/cjs/loader.js:581:15) botium-PluginConnectorContainer at Function.Module._load (internal/modules/cjs/loader.js:507:25) botium-PluginConnectorContainer at Module.require (internal/modules/cjs/loader.js:637:17) botium-PluginConnectorContainer at require (internal/modules/cjs/helpers.js:22:18) botium-PluginConnectorContainer at tryLoadPlugin (D:\backup\sampleBotiumMocha\node_modules\botium-core\src\containers\PluginConnectorContainer.js:16:20) botium-PluginConnectorContainer at Validate.Validate.then (D:\backup\sampleBotiumMocha\node_modules\botium-core\src\containers\PluginConnectorContainer.js:45:29) code: 'MODULE_NOT_FOUND' } +0ms botium-PluginConnectorContainer Botium plugin botium-connector-directline3 loaded +89ms botium-connector-directline3 Validate called +0ms botium-connector-directline3 Build called +1ms botium-connector-directline3 Start called +9ms botium-connector-directline3 Directline Connection Status: 1 / Connecting +24ms botium-bindings-main running testcase1 testCase1.convo.txt +189ms botium-Convo testCase1.convo.txt/Line 3: user says { botium-Convo "sender": "me", botium-Convo "channel": null, botium-Convo "messageText": "insurance claim", botium-Convo "stepTag": "Line 3", botium-Convo "not": false, botium-Convo "asserters": [], botium-Convo "logicHooks": [] botium-Convo } +0ms botium-connector-directline3 UserSays called +12ms botium-connector-directline3 Directline Connection Status: 2 / Online +852ms botium-connector-directline3 Posted activity, assigned ID 3c9AGxY6VhB8nxqDRUpCO1|0000000 +1s botium-Convo testCase1.convo.txt wait for bot null +2s botium-connector-directline3 received message { "type": "message", "id": "3c9AGxY6VhB8nxqDRUpCO1|0000001", "timestamp": "2019-02-01T13:15:04.2137549Z", "localTimestamp": "2019-02-01T13:15:04.07+00:00", "channelId": "directline", "from": { "id": "Ginger_Bot", "name": "Ginger" }, "conversation": { "id": "3c9AGxY6VhB8nxqDRUpCO1" }, "inputHint": "acceptingInput", "attachments": [ { "contentType": "application/vnd.microsoft.card.hero", "content": { "text": "Please contact Vishwanath Gambher to claim insurance.", "images": [ {} ], "tap": { "type": "openUrl", "title": "Insurance" } } } ], "replyToId": "3c9AGxY6VhB8nxqDRUpCO1|0000000" } +9s botium-connector-directline3 received message { "type": "message", "id": "3c9AGxY6VhB8nxqDRUpCO1|0000002", "timestamp": "2019-02-01T13:15:06.1249631Z", "localTimestamp": "2019-02-01T13:15:06.092+00:00", "channelId": "directline", "from": { "id": "Ginger_Bot", "name": "Ginger" }, "conversation": { "id": "3c9AGxY6VhB8nxqDRUpCO1" }, "text": "Is this information helpful?", "inputHint": "expectingInput", "replyToId": "3c9AGxY6VhB8nxqDRUpCO1|0000000" } +22ms botium-Convo testCase1.convo.txt: bot says { botium-Convo "sender": "bot", botium-Convo "sourceData": { botium-Convo "type": "message", botium-Convo "id": "3c9AGxY6VhB8nxqDRUpCO1|0000001", botium-Convo "timestamp": "2019-02-01T13:15:04.2137549Z", botium-Convo "localTimestamp": "2019-02-01T13:15:04.07+00:00", botium-Convo "channelId": "directline", botium-Convo "from": { botium-Convo "id": "Ginger_Bot", botium-Convo "name": "Ginger" botium-Convo }, botium-Convo "conversation": { botium-Convo "id": "3c9AGxY6VhB8nxqDRUpCO1" botium-Convo }, botium-Convo "inputHint": "acceptingInput", botium-Convo "attachments": [ botium-Convo { botium-Convo "contentType": "application/vnd.microsoft.card.hero", botium-Convo "content": { botium-Convo "text": "Please contact Vishwanath Gambher to claim insurance.", botium-Convo "images": [ botium-Convo {} botium-Convo ], botium-Convo "tap": { botium-Convo "type": "openUrl", botium-Convo "title": "Insurance" botium-Convo } botium-Convo } botium-Convo } botium-Convo ], botium-Convo "replyToId": "3c9AGxY6VhB8nxqDRUpCO1|0000000" botium-Convo }, botium-Convo "media": [], botium-Convo "buttons": [], botium-Convo "cards": [ botium-Convo { botium-Convo "image": { botium-Convo "mimeType": "application/unknown" botium-Convo } botium-Convo } botium-Convo ], botium-Convo "messageText": null, botium-Convo "channel": "default" botium-Convo } +9s botium-ScriptingProvider assertBotResponse testCase1.convo.txt/Line 4 (Line 3: #me - insurance claim ) BOT: null = Please contact Vishwanath Gambher to claim insurance. ... +11s botium-bindings-main Test Case "testCase1.convo.txt" failed: { TranscriptError: Error: testCase1.convo.txt/Line 4: Expected bot response (on Line 3: #me - insurance claim ) "null" to match one of "Please contact Vishwanath Gambher to claim insurance." botium-bindings-main at async.waterfall (D:\backup\sampleBotiumMocha\node_modules\botium-core\src\scripting\Convo.js:160:25) botium-bindings-main at D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:473:16 botium-bindings-main at next (D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:5329:29) botium-bindings-main at D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:969:16 botium-bindings-main at runConversation (D:\backup\sampleBotiumMocha\node_modules\botium-core\src\scripting\Convo.js:144:15) botium-bindings-main at async.mapSeries (D:\backup\sampleBotiumMocha\node_modules\botium-core\src\scripting\Convo.js:317:9) botium-bindings-main at D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:1140:9 botium-bindings-main at D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:473:16 botium-bindings-main at iterateeCallback (D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:988:17) botium-bindings-main at D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:969:16 botium-bindings-main at D:\backup\sampleBotiumMocha\node_modules\async\dist\async.js:1137:13 botium-bindings-main at convoStepDone (D:\backup\sampleBotiumMocha\node_modules\botium-core\src\scripting\Convo.js:193:11) botium-bindings-main at scriptingEvents.onBotStart.then.then (D:\backup\sampleBotiumMocha\node_modules\botium-core\src\scripting\Convo.js:269:28) botium-bindings-main at process._tickCallback (internal/process/next_tick.js:68:7) botium-bindings-main name: 'TranscriptError', botium-bindings-main transcript: botium-bindings-main Transcript { botium-bindings-main steps: [ [TranscriptStep], [TranscriptStep] ], botium-bindings-main scriptingMemory: {}, botium-bindings-main convoBegin: 2019-02-01T13:14:59.846Z, botium-bindings-main convoEnd: 2019-02-01T13:15:10.529Z, botium-bindings-main err: botium-bindings-main Error: testCase1.convo.txt/Line 4: Expected bot response (on Line 3: #me - insurance claim ) "null" to match one of "Please contact Vishwanath Gambher to claim insurance." botium-bindings-main at ScriptingProvider.assertBotResponse (D:\backup\sampleBotiumMocha\node_modules\botium-core\src\scripting\ScriptingProvider.js:76:17) botium-bindings-main at scriptingEvents.onBotStart.then.then (D:\backup\sampleBotiumMocha\node_modules\botium-core\src\scripting\Convo.js:267:42) botium-bindings-main at process._tickCallback (internal/process/next_tick.js:68:7) } } +11s 1) testCase1.convo.txt botium-connector-directline3 Stop called +29ms botium-connector-directline3 unsubscribing from directline activity subscription +1ms botium-connector-directline3 unsubscribing from directline connectionstatus subscript

codeforequity-at commented 5 years ago

the newest build is 0.0.6 https://www.npmjs.com/package/botium-connector-directline3

Nimish121 commented 5 years ago

Thanks its working fine with latest build i.e. 0.0.6 . I also identified the below mentioned scenarios while execution my test cases: Scenario 1:If the bot replies in multiple lines (more than one line), in automation its validate only first line . Suppose below is the bot reply :

me

insurance claim

bot

Please contact Vishwanath Gambher to claim insurance. Is this information helpful?

At validation through automation bot will reply only first line i.e. Please contact Vishwanath Gambher to claim insurance.

Scenario 2: While executing multiple test cases , After execution of first test case bot say nothing.

Below are the log details:

1) Botium Test Suite for botiumdirectline testCase2.convo.txt: TranscriptError: Error: testCase2.convo.txt/Line 3: Expected bot response (on Line 1: #me - insurance claim ) "Please contact Vishwanath Gambher to claim insurance." to match one of "Please contact Vishwanath Gambher to claim insurance. Is this information helpful?" at async.waterfall (node_modules\botium-core\src\scripting\Convo.js:160:25) at D:\backup\botiumDirectLine\node_modules\async\dist\async.js:473:16 at next (node_modules\async\dist\async.js:5329:29) at D:\backup\botiumDirectLine\node_modules\async\dist\async.js:969:16 at runConversation (node_modules\botium-core\src\scripting\Convo.js:144:15) at async.mapSeries (node_modules\botium-core\src\scripting\Convo.js:317:9) at D:\backup\botiumDirectLine\node_modules\async\dist\async.js:1140:9 at D:\backup\botiumDirectLine\node_modules\async\dist\async.js:473:16 at iterateeCallback (node_modules\async\dist\async.js:988:17) at D:\backup\botiumDirectLine\node_modules\async\dist\async.js:969:16 at D:\backup\botiumDirectLine\node_modules\async\dist\async.js:1137:13 at convoStepDone (node_modules\botium-core\src\scripting\Convo.js:193:11) at scriptingEvents.onBotStart.then.then (node_modules\botium-core\src\scripting\Convo.js:269:28) at process._tickCallback (internal/process/next_tick.js:68:7)

2) Botium Test Suite for botiumdirectline testCase3.convo.txt: TranscriptError: Error: testCase3.convo.txt/Line 3: bot says nothing at async.waterfall (node_modules\botium-core\src\scripting\Convo.js:160:25) at D:\backup\botiumDirectLine\node_modules\async\dist\async.js:473:16 at next (node_modules\async\dist\async.js:5329:29) at D:\backup\botiumDirectLine\node_modules\async\dist\async.js:969:16 at runConversation (node_modules\botium-core\src\scripting\Convo.js:144:15) at async.mapSeries (node_modules\botium-core\src\scripting\Convo.js:317:9) at D:\backup\botiumDirectLine\node_modules\async\dist\async.js:1140:9 at D:\backup\botiumDirectLine\node_modules\async\dist\async.js:473:16 at iterateeCallback (node_modules\async\dist\async.js:988:17) at D:\backup\botiumDirectLine\node_modules\async\dist\async.js:969:16 at D:\backup\botiumDirectLine\node_modules\async\dist\async.js:1137:13 at convoStepDone (node_modules\botium-core\src\scripting\Convo.js:193:11) at scriptingEvents.onBotStart.then.then (node_modules\botium-core\src\scripting\Convo.js:253:24)

codeforequity-at commented 5 years ago

For scenario 1, the assertions for the bot response depend on how your bot delivers the response. maybe you have to use other asserters. logfile would help here.

scenario 2: depending on the chatbot implementation, it sometimes happens that a Botium conversation is just continued by the chatbot, no new conversation started, locked in the conversation state. we've had such trouble several times before. maybe the same here, and logfile would help.

stale[bot] commented 5 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.