aws-samples / aws-lex-web-ui

Sample Amazon Lex chat bot web interface
Other
734 stars 466 forks source link

Method#3 HTML and markdown support #180

Closed MeerZaheen closed 6 months ago

MeerZaheen commented 4 years ago

I'm using method #3: "Use the pre-built libraries from the dist directory of this repo"

does this support HTML and markdown?

I tried to have ""AllowSuperDangerousHTMLInMessage": true," in my lex-web-ui-loader.json file.

it does not work.

bobpskier commented 4 years ago

Yes, this should work. Are you using a CustomPayload defined in Lex Console or using the earlier technique of supplying html or markdown in a Session attribute named appContext? The information at https://github.com/aws-samples/aws-lex-web-ui/blob/master/lex-web-ui/README.md#markdown-and-html-support is relevant.

MeerZaheen commented 4 years ago

Yes, this should work. Are you using a CustomPayload defined in Lex Console or using the earlier technique of supplying html or markdown in a Session attribute named appContext? The information at https://github.com/aws-samples/aws-lex-web-ui/blob/master/lex-web-ui/README.md#markdown-and-html-support is relevant.

I'm using custom payload defined in my lambda. The lambda queries the results from dynamoDB and outputs the result in html which is then displayed in the web ui as HTML (allowsuperdangeroudHTMLinmessage" activated.

bobpskier commented 4 years ago

Here are some things to validate.

First is to verify how your lambda is returning content. The following is an example nodejs based lambda handler that returns a custom payload that renders either markdown or html.

let lexResponseMarkdown = {
    sessionAttributes: {
        ExampleHandler: "true"
    },
    dialogAction: {
        type: "Close",
        fulfillmentState: "Fulfilled",
        message: {
            contentType: "CustomPayload",
            content: "# Markdown Payload Header \n ## Markdown Payload H2 \n *This is bold*"
        },
    }
}

let lexResponseHtml = {
    sessionAttributes: {
        ExampleHandler: "true"
    },
    dialogAction: {
        type: "Close",
        fulfillmentState: "Fulfilled",
        message: {
            contentType: "CustomPayload",
            content: "<h1> HTML Payload Header </h1> </h2> HTML Payload H2 </h2>  <p> <strong>This is bold from HTML strong</strong> </p>"
        },
    }
}

let errorMessage = {
    sessionAttributes: {
        ExampleHandler: "true"
    },
    dialogAction: {
        type: "Close",
        fulfillmentState: "Fulfilled",
        message: {
            contentType: "CustomPayload",
            content: "Sorry, I could not process your message"
        },
    }
}

exports.lambdaHandler = async (event, context) => {
    console.log("event: " + JSON.stringify(event,null,2));
    if (event.currentIntent.name === 'MarkdownAsCustomResourceFromLambda') {
        console.log("returning:" + JSON.stringify(lexResponseMarkdown));
        return lexResponseMarkdown;
    } else if (event.currentIntent.name === 'HTMLAsCustomResourceFromLambda') {
        console.log("returning:" + JSON.stringify(lexResponseHtml));
        return lexResponseHtml;
    } else {
        return errorMessage;
    }
};

Next is to validate that the config object being served up to the browser client has the correct setting. Are you passing a config object via the html page that has this configuration or are you specifying the configuration setting in the lex-web-ui-loader-config.json file which served from the web server? If using a server based approach, you can validate the setting using the debug/network console and observe the results of the network request for lex-web-ui-loader-config.json file.

Can you send me a link to an example lex-web-ui that is failing along with sample input to the bot to use?

The following link demonstrates the behavior working as intended.

https://lex-web-ui-md-test-codebuilddeploy-1-webappbucket-6wm6m6awpgwi.s3.amazonaws.com/parent.html

Provide the input "show me how lambda generates markdown" or "show me how lambda generates html". These will supply a CustomPayload using the lambda above using either html or markdown.

I'll leave this sample running for a bit.

MeerZaheen commented 4 years ago

Will try this today

MeerZaheen commented 4 years ago

image

index.jade:

doctype html html head meta(charset='utf-8') meta(name='viewport', content='width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no, minimal-ui') title "" // empty favicon to avoid 404 link(href='data:image/x-icon;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQEAYAAABPYyMiAAAABmJLR0T///////8JWPfcAAAACXBIWXMAAABIAAAASABGyWs+AAAAF0lEQVRIx2NgGAWjYBSMglEwCkbBSAcACBAAAeaR9cIAAAAASUVORK5CYII=', rel='icon', type='image/x-icon') body // babel-polyfill needed for IE11 script(src='https://cdnjs.cloudflare.com/ajax/libs/babel-polyfill/6.26.0/polyfill.min.js', crossorigin='anonymous', integrity='sha256-WRc/eG3R84AverJv0zmqxAmdwQxstUpqkiE+avJ3WSo=') // Loader script. Creates a global variable named ChatbotUiLoader. // LexWebUi loader script(src='./lex-web-ui-loader.js') script. // loader options control how the loader obtains the config and // dependencies. It can be optionally passed as an argument to the // FullPageLoader constructor to override the defaults // var loaderOptions = { // shouldLoadConfigFromMobileHubFile: false, // configUrl: './chatbot-ui-loader-config-2.json' // }; // instantiate the loader var lexWebUiLoader = new ChatBotUiLoader.FullPageLoader(); // this config is used to pass run-time configuration to the chatbot UI // as an argument to the load() function. The config passed through // load() is merged with the config from other sources and overrides // their values // var chatbotUiconfig = { // cognito: { // poolId: '' // }, // lex: { // initialText: '', // botName: '' // }, // ui: { // toolbarTitle: '', // toolbarLogo: '' // }, // }; // You can pass the config as a parameter. // The loader can also get its config dynamically // from a JSON file or from the Mobile Hub config script lexWebUiLoader.load() // returns a promise that is resolved once the chatbot UI is loaded .then(function () { console.log('chatbot UI loaded'); }) .catch(function (error) { console.error('chatbot UI failed to load', error); });

lex-web-ui-loader-config.json

image

LAMBDA output += "

"; data.Items.forEach(function(item) { count++; if (count%5 == 0) { output += "\n\n\n"; } else { output += "\n"; } //output += "";

                    if (count === data.Items.length) {
                        output += "\n</table>";
                        console.log("\n\nAnd now finalOutput is: " + output + "\n\n");

                        return resolve({
                            sessionAttributes,
                            dialogAction: {
                                type: 'ElicitIntent',
                                "message": {
                                    "contentType": "CustomPayload",
                                    "content": output
                                }
                            }
                        });
                    }

image

MeerZaheen commented 4 years ago

Above is my setup.

As i have "allowsuperdangeroudHTMLinmessage" set to true, I still do not receive the output in HTML format.

It looks like my aws-config.js and lex-web-ui-loader-config.json is returning 304 - (not modified)where as in your setup, you get 200 - (ok)

image

bobpskier commented 4 years ago

The 304 is an indication that the browser will use a previously cached value for the object. If you clear cache or in chrome debugger on network tab reload with cache disabled, does the config file indicate the proper settings?

Also in the returned dialogAction can your try type of "Close" with fulfillmentState of "Fulfilled"? Is there a reason you are returning an ElicitIntent for table data to be displayed?

    dialogAction: {
        type: "Close",
        fulfillmentState: "Fulfilled",
bobpskier commented 4 years ago

I did confirm that returning an ElicitIntent with a custom payload should work as well and display either markdown or html hence that should not be the cause of the problem.

MeerZaheen commented 4 years ago

I have the same configuration for lex-web-ui component but when I'm using method #3: "Use the pre-built libraries from the dist directory of this repo", it is not working with the same configuration. I'm not too sure why. Would I be able to speak to you privately?

bobpskier commented 4 years ago

Any chance you'll be at re:Invent next week? I could provide some one on one time there to look at it.

Are you using the "running locally" approach to test it out? Can you give the following a test first.

Make sure your local system is running nodejs 10.16.1 and npm 6.13.1.

Create a new folder to test in and clone the lex-web-ui GitHub repo.

cd 'newfolder'
git clone https://github.com/aws-samples/aws-lex-web-ui.git

Then modify three fields in aws-lex-web-ui/src/config/lex-web-ui-loader-config.json

"cognito": {
    "poolId": "fill in with your cognito identity pool id",
...
},
  "lex": {
    "botName": "fill in with your bot name",
...
}
  "ui": {
...
    "AllowSuperDangerousHTMLInMessage": true,
...
}

Then execute the following

cd aws-lex-web-ui
npm install
cd lex-web-ui
npm install
cd ../build
./release.sh
cd ..
npm start

This should fire up a web server on localhost on port 8000. You can test this to see if the behavior is correct with respect to html/markdown.

MeerZaheen commented 4 years ago

Any chance you'll be at re:Invent next week? I could provide some one on one time there to look at it.

Are you using the "running locally" approach to test it out? Can you give the following a test first.

Make sure your local system is running nodejs 10.16.1 and npm 6.13.1.

Create a new folder to test in and clone the lex-web-ui GitHub repo.

cd 'newfolder'
git clone https://github.com/aws-samples/aws-lex-web-ui.git

Then modify three fields in aws-lex-web-ui/src/config/lex-web-ui-loader-config.json

"cognito": {
    "poolId": "fill in with your cognito identity pool id",
...
},
  "lex": {
    "botName": "fill in with your bot name",
...
}
  "ui": {
...
    "AllowSuperDangerousHTMLInMessage": true,
...
}

Then execute the following

cd aws-lex-web-ui
npm install
cd lex-web-ui
npm install
cd ../build
./release.sh
cd ..
npm start

This should fire up a web server on localhost on port 8000. You can test this to see if the behavior is correct with respect to html/markdown.

I'm in Toronto, Canada and thus, wont be able to join :( I'll try this approach and get back to you soon

MeerZaheen commented 4 years ago

Any chance you'll be at re:Invent next week? I could provide some one on one time there to look at it.

Are you using the "running locally" approach to test it out? Can you give the following a test first.

Make sure your local system is running nodejs 10.16.1 and npm 6.13.1.

Create a new folder to test in and clone the lex-web-ui GitHub repo.

cd 'newfolder'
git clone https://github.com/aws-samples/aws-lex-web-ui.git

Then modify three fields in aws-lex-web-ui/src/config/lex-web-ui-loader-config.json

"cognito": {
    "poolId": "fill in with your cognito identity pool id",
...
},
  "lex": {
    "botName": "fill in with your bot name",
...
}
  "ui": {
...
    "AllowSuperDangerousHTMLInMessage": true,
...
}

Then execute the following

cd aws-lex-web-ui
npm install
cd lex-web-ui
npm install
cd ../build
./release.sh
cd ..
npm start

This should fire up a web server on localhost on port 8000. You can test this to see if the behavior is correct with respect to html/markdown.

Yes this fires up a web server on localhost 8080 and the behavior is correct with respect to HTML/markdown

jeremywilhelm commented 2 years ago

We are having the same issue with markdown and html not working in live bot deployments despite ""AllowSuperDangerousHTMLInMessage": true,"

Was there ever a resolution to this that perhaps didn't get posted here?

bobpskier commented 2 years ago

@jeremywilhelm If you've doubled checked your config, usually, the problem has to do with the response being provided to lex-web-ui. See https://github.com/aws-samples/aws-lex-web-ui/blob/master/lex-web-ui/README.md#markdown-and-html-support. Can you post the response json captured through your browser's debug/network tab for the post request to the bot?

jeremywilhelm commented 2 years ago

This is what comes back if viewed in the inspector via Chrome:

{"interpretations":[{"intent":{"confirmationState":"None","name":"ModifyHQInfo","slots":{},"state":"ReadyForFulfillment"},"nluConfidence":{"score":0.84}},{"intent":{"name":"SpeakToRepresentative","slots":{}},"nluConfidence":{"score":0.32}},{"intent":{"name":"RegisterCompany","slots":{}},"nluConfidence":{"score":0.32}},{"intent":{"name":"UpdateCompanyContacts","slots":{}},"nluConfidence":{"score":0.27}},{"intent":{"name":"LoginTechnicalIssues","slots":{"HaveAccount":null}},"nluConfidence":{"score":0.26}}],"messages":[{"content":"**You can update your company's information** (like name, address, XXX, XXXXX code) in the system). Log on at [the Sign-in Page](https://XXX.org/XXX/signin) and select your company in <strong>Your Company List</strong>. On the Company Dashboard, under <strong>XXX Reporting, Confirm Company Details</strong>, you can update your company's information.","contentType":"PlainText"}],"sessionId":"us-east-1:fbcbcc9c-2014-4986-9c14-2e381c177cf2","sessionState":{"dialogAction":{"type":"Close"},"intent":{"confirmationState":"None","name":"ModifyHQInfo","slots":{},"state":"ReadyForFulfillment"},"originatingRequestId":"b877b354-ecef-49f2-9f9e-892dc25bd793","sessionAttributes":{"userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.63 Safari/537.36 Edg/102.0.1245.33"}}}

bobpskier commented 2 years ago

@jeremywilhelm The messages array in the data above has a single message with contentType of "PlainText". This alone is not sufficient to support markdown or html formatting in lex-web-ui. Whatever is generating the response, I'm guessing a Lambda function, needs to include a session attribute element containing the markdown or html content to be displayed. It would be something similar to the following. appContext below is a stringified version of a json object.

"sessionState": {
    "sessionAttributes": {
      "appContext": "{\"altMessages\":{\"markdown\":\"**You can update your company's information** (like name, address, XXX, XXXXX code) in the system). Log on at [the Sign-in Page](https://XXX.org/XXX/signin) and select your company in <strong>Your Company List</strong>. On the Company Dashboard, under <strong>XXX Reporting, Confirm Company Details</strong>, you can update your company's information.\"}}"
      },
      "dialogAction": {
      "type": "Close"
      },
      "intent": {
        "confirmationState": "None",
        "name": "ModifyHQInfo",
        "slots": {},
        "state": "ReadyForFulfillment"
      },
    }
  }    

The original messages array element with the PlainText content should contain just plaintext. LexWebUi will properly format markdown from the session attribute. Other channels that do not support markdown would use just the plaintext. Hope this helps.

jeremywilhelm commented 2 years ago

That got us on the right path! Thanks for the help.

" + item.NAME.S + " " + item.NAME.S + "