Beckn-gemini-bot - Enhancements

[emmayank]

Description

Enhance the existing AI bot Lisa in Beckn-Gemini with several key improvements. These enhancements will make Lisa more robust, flexible, and capable of handling various user inputs while maintaining the context of the conversation. The following features are to be implemented:

1. Switching Between Flow Without Breaking: The bot should allow users to switch between different conversation flows without disrupting the ongoing flow. For instance, while in the consumer flow, users can ask unrelated questions, and the bot will handle them while returning to the original flow. 117

2. Adding Retrial Mechanism in Case of Failure: Implement a mechanism where the bot retries certain actions in case of errors or failures in the API/model, ensuring a more reliable experience and reducing disruptions in conversation 116.

3. Adding Support for Voice/Image/Video Input: Since Lisa is designed to be multimodal, the bot should support voice, image, and video inputs. Users should be able to switch between these input modes at any point in the conversation. 115

4. Controlling JSON/Code Output: Prevent the bot from returning JSON objects, variable names, or code snippets as responses, ensuring that all outputs are user-friendly and conversational. 114

5. Preserving Chat Context: The bot should preserve the context of conversations across multiple interactions, allowing users to ask follow-up questions without having to restate their previous input. 113

Goals

• [ ] Enable the bot to switch between different conversation flows without breaking the original flow.
• [ ] Implement a retrial mechanism to recover from API/model errors or failures.
• [ ] Add support for multimodal inputs (voice, image, video) to allow flexible user interactions.
• [ ] Ensure that the bot does not return JSON objects or code snippets in its responses.
• [ ] Preserve conversation context so the bot can handle follow-up questions without losing the thread of the previous conversation.

Expected Outcome

• A more robust and flexible AI bot that can handle flow switching, multimodal inputs, and retry mechanisms.
• Improved user experience by ensuring that the bot can recover from errors and respond contextually without exposing internal code or logic.
• Users will have the ability to engage with the bot using different input formats (voice, image, video) and expect consistent responses.

Acceptance Criteria

• [ ] The bot successfully allows users to switch between conversation flows without breaking the original flow.
• [ ] The retrial mechanism is implemented and works to handle API/model failures.
• [ ] The bot supports voice, image, and video inputs and can respond accordingly.
• [ ] JSON objects or code snippets are never returned in responses, ensuring conversational outputs.
• [ ] The bot preserves the context of previous conversations, enabling smooth follow-up queries.

Mockups / Wireframes

NA

Product Name

Beckn-Gemini Bot (Lisa)

Domain

Conversational AI / Multimodal Input / Error Handling

Tech Skills Needed

• Conversation Flow Management
• Multimodal Input Processing (Voice, Image, Video)
• Error Handling and Retry Mechanism
• Context Preservation in AI
• JavaScript/TypeScript (or relevant language)

Complexity

High

Category

Bot Enhancement

Sub Category

Multimodal Interaction / Error Handling / Context Management

Project View

Beckn-Gemini Bot

Project Name

Beckn-Gemini Bot Enhancements

[emmayank]

Current Progress

1. @shreyvishal has started exploring blogs/documentation for preserving chat context
2. Had a session with monojith on the correct approach
3. Did a POC for preserving context

Next Step

1. To demonstrate the POC, on 14th October'24
2. Proceed to incorporate the changes in the existing codebase
3. Move to next point