codeforequity-at / botium-core

The Selenium for Chatbots - Bots Testing Bots
https://www.botium.ai
MIT License
230 stars 64 forks source link

How can the Microsoft adaptive cards be tested in the Generic HTTP/JSON Technology in the latest EE Botium box #351

Closed Aishwarya-J-S closed 4 years ago

Aishwarya-J-S commented 5 years ago

I have been testing the Chat bot with the payload which has the attachment type as button as well media which is working good.

Now that I need to test the Microsoft adaptive cards with my Chat bot. When there is a adaptive card attached to my payload, I am getting the following error in the backend while testing it with the live Chat feature of the Chat bot. The page becomes blank when it identifies an adaptive card in the payload.

2019-08-12T14:34:28.580Z botium-box-server-livechat updateConversation(405a91e0-bd0e-11e9-aaba-55bfcf69dc8f) TTL(seconds): 300
2019-08-12T14:34:28.583Z botium-box-server-livechat updateConversation(405a91e0-bd0e-11e9-aaba-55bfcf69dc8f) TTL(seconds): 300
2019-08-12T14:34:28.891Z botium-box-server-livechat Received IPC message LOG from worker. conversationId: 405a91e0-bd0e-11e9-aaba-55bfcf69dc8f, message: 'Received IPC message STOP_CONVERSATION from worker. Message: undefined'
2019-08-12T14:34:28.891Z botium-box-server-livechat-worker Received IPC message STOP_CONVERSATION from worker. Message: undefined
2019-08-12T14:34:28.892Z botium-BaseContainer Cleanup rimrafing temp dir /app/server/botiumwork/defaultproject-20190812-143412-E5OFs
2019-08-12T14:34:28.897Z botium-box-server-livechat Received IPC message LOG from worker. conversationId: 405a91e0-bd0e-11e9-aaba-55bfcf69dc8f, message: 'stopConversation(405a91e0-bd0e-11e9-aaba-55bfcf69dc8f) container stopped.'
2019-08-12T14:34:28.897Z botium-box-server-livechat-worker stopConversation(405a91e0-bd0e-11e9-aaba-55bfcf69dc8f) container stopped.
2019-08-12T14:34:28.897Z botium-box-server-livechat Received IPC message STOP_CONVERSATION_FINISHED from worker. conversationId: 405a91e0-bd0e-11e9-aaba-55bfcf69dc8f, message: { conversationId: '405a91e0-bd0e-11e9-aaba-55bfcf69dc8f' }
codeforequity-at commented 5 years ago

Please send more of the logs, this only covers the part where the live chat already failed, as far as I can see.

codeforequity-at commented 5 years ago

Received the logfile from email. The JSONPath setting for extracting the Buttons are not matching the received document. Can you please attach your botium capabilities settings to further analyze.

Aishwarya-J-S commented 5 years ago

Sent the Capabilities file. The buttons are identified by Botium for other responses in the same Chat bot.

codeforequity-at commented 5 years ago

So I can see why this is happening. The capabilitiy SIMPLEREST_BUTTONS_JSONPATH is set to $..data[], which means: "when finding an element named data, use the first child element as button text". This works well for the nested actions, but not for the adaptive cards, as they are wrapped within a data element as well. My suggestions is to rewrite the SIMPLEREST_BUTTONS_JSONPATH-capability to $..actions[].title, it will resolve to these buttons then:

   "Laptop",
   "Software",
   "Accessories"
]

Here is the full structure from your HTTP response, you can easily test JSONPath expressions here: https://jsonpath.curiousconcept.com/

{
   "action": {},
   "text": "",
   "contentType": "plainText",
   "attachments": [
      {
         "data": [
            {
               "content": {
                  "body": [
                     {
                        "text": "Ok, Now please select a category.",
                        "type": "TextBlock"
                     },
                     {
                        "type": "ColumnSet",
                        "columns": [
                           {
                              "spacing": "None",
                              "type": "Column",
                              "width": 3,
                              "items": [
                                 {
                                    "type": "ColumnSet",
                                    "columns": [
                                       {
                                          "items": [
                                             {
                                                "url": "<url>",
                                                "horizontalAlignment": "Center",
                                                "spacing": "Small",
                                                "type": "Image",
                                                "size": "Large"
                                             }
                                          ],
                                          "type": "Column",
                                          "width": 3
                                       }
                                    ]
                                 }
                              ]
                           }
                        ]
                     }
                  ],
                  "$schema": "<url>",
                  "version": "1.0",
                  "type": "AdaptiveCard",
                  "actions": [
                     {
                        "data": {
                           "category": "laptop"
                        },
                        "type": "Action.Submit",
                        "title": "Laptop"
                     }
                  ]
               },
               "contentType": "application/vnd.microsoft.card.adaptive"
            },
            {
               "content": {
                  "body": [
                     {
                        "text": "Ok, Now please select a category.",
                        "type": "TextBlock"
                     },
                     {
                        "type": "ColumnSet",
                        "columns": [
                           {
                              "spacing": "None",
                              "type": "Column",
                              "width": 3,
                              "items": [
                                 {
                                    "type": "ColumnSet",
                                    "columns": [
                                       {
                                          "items": [
                                             {
                                                "url": "<url>",
                                                "horizontalAlignment": "Center",
                                                "spacing": "Small",
                                                "type": "Image",
                                                "size": "Large"
                                             }
                                          ],
                                          "type": "Column",
                                          "width": 3
                                       }
                                    ]
                                 }
                              ]
                           }
                        ]
                     }
                  ],
                  "$schema": "<url>",
                  "version": "1.0",
                  "type": "AdaptiveCard",
                  "actions": [
                     {
                        "data": {
                           "category": "software"
                        },
                        "type": "Action.Submit",
                        "title": "Software"
                     }
                  ]
               },
               "contentType": "application/vnd.microsoft.card.adaptive"
            },
            {
               "content": {
                  "body": [
                     {
                        "text": "Ok, Now please select a category.",
                        "type": "TextBlock"
                     },
                     {
                        "type": "ColumnSet",
                        "columns": [
                           {
                              "spacing": "None",
                              "type": "Column",
                              "width": 3,
                              "items": [
                                 {
                                    "type": "ColumnSet",
                                    "columns": [
                                       {
                                          "items": [
                                             {
                                                "url": "<url>",
                                                "horizontalAlignment": "Center",
                                                "spacing": "Small",
                                                "type": "Image",
                                                "size": "Large"
                                             }
                                          ],
                                          "type": "Column",
                                          "width": 3
                                       }
                                    ]
                                 }
                              ]
                           }
                        ]
                     }
                  ],
                  "$schema": "<url>",
                  "version": "1.0",
                  "type": "AdaptiveCard",
                  "actions": [
                     {
                        "data": {
                           "category": "Accessories"
                        },
                        "type": "Action.Submit",
                        "title": "Accessories"
                     }
                  ]
               },
               "contentType": "application/vnd.microsoft.card.adaptive"
            }
         ],
         "type": "carousel"
      }
   ],
   "trace": {
      "lang": "en",
      "entities": [],
      "intent": {},
      "input": "Order Item",
      "mod-input": "Order Item",
      "behav-analysis": {
         "sentiment": "Positive"
      }
   }
}
Aishwarya-J-S commented 5 years ago

When I give the SIMPLEREST_BUTTONS_JSONPATH as $..actions[].title, it will resolve the issue with the responses having adaptive cards, but not for the responses which has the buttons in the $..data[]

Can multiple capability of SIMPLEREST_BUTTONS_JSONPATH be provided? When I tried, it takes only capability which has the initially provided.

Any possibility for it be added with some other capability?

codeforequity-at commented 5 years ago

yes. just rename the additional capabilities and append 0, 1 and so on, it will search for buttons in different locations then.

Aishwarya-J-S notifications@github.com schrieb am Mi., 14. Aug. 2019, 13:53:

When I give the SIMPLEREST_BUTTONS_JSONPATH as $..actions[].title, it will resolve the issue with the responses having adaptive cards, but not for the responses which has the buttons in the $..data[]

Can multiple capability of SIMPLEREST_BUTTONS_JSONPATH be provided? When I tried, it takes only capability which has the initially provided.

Any possibility for it be added with some other capability?

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/codeforequity-at/botium-core/issues/351?email_source=notifications&email_token=AGIR2T2RRV3GGVJJAOLXJFDQEPW4NA5CNFSM4ILB6DR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4IRYPA#issuecomment-521215036, or mute the thread https://github.com/notifications/unsubscribe-auth/AGIR2T7GBZYPUYXOGUI3DKLQEPW4NANCNFSM4ILB6DRQ .

Aishwarya-J-S commented 5 years ago

Do you mean to append it as following image It didnot work for me

codeforequity-at commented 5 years ago

for the first one, dont append anything, only for the additional ones (i know, not optimal, added it to our backlog to provide a better solution)

Aishwarya-J-S notifications@github.com schrieb am Mi., 14. Aug. 2019, 14:28:

Do you mean to append it as following [image: image] https://user-images.githubusercontent.com/49233293/63020963-1744ec00-bebd-11e9-9622-d466937bd71b.png It didnot work for me

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/codeforequity-at/botium-core/issues/351?email_source=notifications&email_token=AGIR2T3G42GFF5TXZKQQHYDQEP3APA5CNFSM4ILB6DR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4IUK6A#issuecomment-521225592, or mute the thread https://github.com/notifications/unsubscribe-auth/AGIR2T5WPQKG4V3ANQLYYILQEP3APANCNFSM4ILB6DRQ .

Aishwarya-J-S commented 5 years ago

I have added the Capability as follows but still it takes only one value

image

It works only for SIMPLEREST_BUTTONS_JSONPATH0 and not for the SIMPLEREST_BUTTONS_JSONPATH Any alternative solution?

codeforequity-at commented 5 years ago

no, this should actually work. logs please ...

codeforequity-at commented 5 years ago

It comes into my mind that maybe for your project you need some better mechanism than JSONpath. It is possible to hook your custom functionality developed in Javascript into the HTTP/JSON connector:

https://botium.atlassian.net/wiki/spaces/BOTIUM/pages/24510469/Generic+HTTP+S+JSON+Connector#SIMPLEREST_RESPONSE_HOOK

Aishwarya-J-S commented 5 years ago

I have a response as below where I have the following entities as below

  1. text

    "text": "Below are the Frequently Asked questions on Outlook. Please select one of them or type your query below."

  2. drop down "choices": [ { "value": "Can I access Outlook on personal laptop outside my workplace network?", "title": "Can I access Outlook on personal laptop outside my workplace network?" }, { "value": "How to set Out of Office mail in Outlook?", "title": "How to set Out of Office mail in Outlook?" }, { "value": "How to configure group id in outlook?", "title": "How to configure group id in outlook?" }, { "value": "How to recall an email in Outlook?", "title": "How to recall an email in Outlook?" } ] } ]

  3. Button

    { "type": "Action.Submit", "title": "Ask" }

The whole response i get from bot is as follows.

{ "action": {}, "text": "", "attachments": [ { "data": [ { "content": { "body": [ { "wrap": true, "text": "Below are the Frequently Asked questions on Outlook. Please select one of them or type your query below.", "type": "TextBlock" }, { "type": "Input.ChoiceSet", "id": "FAQ_Outlook", "value": "1", "choices": [ { "value": "Can I access Outlook on personal laptop outside my workplace network?", "title": "Can I access Outlook on personal laptop outside my workplace network?" }, { "value": "How to set Out of Office mail in Outlook?", "title": "How to set Out of Office mail in Outlook?" }, { "value": "How to configure group id in outlook?", "title": "How to configure group id in outlook?" }, { "value": "How to recall an email in Outlook?", "title": "How to recall an email in Outlook?" } ] } ], "$schema": "http://adaptivecards.io/schemas/adaptive-card.json", "version": "1.0", "type": "AdaptiveCard", "actions": [ { "type": "Action.Submit", "title": "Ask" } ] }, "contentType": "application/vnd.microsoft.card.adaptive" } ], "type": "AdaptiveCard" } ], "trace": { "lang": "en", "entities": [], "intent": {}, "input": "Outlook", "mod-input": "Outlook", "behav-analysis": { "sentiment": "Positive" } } }

I want to fetch the mentioned entities from the whole response and write test cases for the same. How can I do that?

codeforequity-at commented 5 years ago

This will be part of the next build of the Holmes connector.