Changing the LLM_CHAT_HISTORY_MAX_MESSAGES in QnABot caused error with processing utterances

[ganapras]

I was working with LCA deployment ingesting from Connect. Both the live agent assist(with LLM based suggestions, no programmed qna) and the agent assist search bot were working fine. I changed the LLM_CHAT_HISTORY_MAX_MESSAGES value to non default and the live suggestions based on utterances either by LLM or through pre programmed qna stopped working. The typed search continued to work.

I debugged and found this error response for every utterance request coming through in the fulfillment lambda cloudwatch logs .

2023-12-14T00:25:48.668Z    cbe86e8d-efa1-4d3f-bccf-457961bc9817    INFO    {
    "StatusCode": 200,
    "FunctionError": "Unhandled",
    "ExecutedVersion": "$LATEST",
    "Payload": "{\"errorMessage\": \"bad operand type for unary -: 'str'\", \"errorType\": \"TypeError\", \"requestId\": \"02cb3455-ba0d-4bfd-b278-a5c627d1baeb\", \"stackTrace\": [\"  File \\\"/var/task/index.py\\\", line 47, in handler\\n    event = format_response(event, transcript)\\n\", \"  File \\\"/var/task/index.py\\\", line 28, in format_response\\n    transcriptSegments = transcriptSegments[-maxMessages:]\\n\"]}"
}

On further investigating the QNA-FetchTranscript-LCA-svg I found this error:

2023-12-13T13:04:31.608-08:00   Using last 14 conversation turns (LLM_CHAT_HISTORY_MAX_MESSAGES)
2023-12-13T13:04:31.609-08:00
Copy
[ERROR] TypeError: bad operand type for unary -: 'str'
Traceback (most recent call last):
  File "/var/task/index.py", line 47, in handler
    event = format_response(event, transcript)
  File "/var/task/index.py", line 28, in format_response
    transcriptSegments = transcriptSegments[-maxMessages:]
[ERROR] TypeError: bad operand type for unary -: 'str' Traceback (most recent call last):   File "/var/task/index.py", line 47, in handler     event = format_response(event, transcript)   File "/var/task/index.py", line 28, in format_response     transcriptSegments = transcriptSegments[-maxMessages:]

The issue was exposed by the fact that I modified the setting in QnABot for LLM_CHAT_HISTORY_MAX_MESSAGES.. This caused the setting to re re-stored as a string, not an int. Which exposed a bug in the code in lines maxMessages = event["req"]["_settings"].get("LLM_CHAT_HISTORY_MAX_MESSAGES", 20) and transcriptSegments = transcriptSegments[-maxMessages:] which do not defend against the possibility that LLM_CHAT_HISTORY_MAX_MESSAGES is a string.. The fix is to explicitly cast maxMessages as int.. eg maxMessages = int(event["req"]["_settings"].get("LLM_CHAT_HISTORY_MAX_MESSAGES", 20))

[rstrahan]

Thanks! Fixed in PR #122 - will be in next release.