[Bug] Image understanding and paraphrase accuracy

[dezhouc2]

Describe the bug

Sorry for the misunderstanding (not really a bug). Would it be possible to increase the number of tokens? Currently, 4096 tokens are not enough because I am trying to implement few-shot chain of thought within the prompt. Additionally, the knowledge database's paraphrasing capability is sometimes poor. It does not capture the question when an alternative form of the question is asked. See the example below.

Original question: How to change my user settings?

original answer:

Unfortunately, the contexts provided do not contain any information about how to change user settings on the Uptrillion Merchant Portal. The contexts cover topics such as managing users, roles, transactions, and profile settings, but do not specifically mention how to change user settings.Without any relevant information in the given contexts, I do not have enough information to provide the steps for changing user settings on the portal. The documentation provided does not seem to cover this particular functionality.If you have any additional context or documentation that covers how to change user settings on the Uptrillion Merchant Portal, I would be happy to review that and provide a more detailed response. However, based solely on the information provided in the current contexts, I do not have enough information to describe the steps for changing user settings.

Alternative question: How should I check my profile?

alternative answer:

According to the contexts provided, to check your profile on the Uptrillion merchant portal, you can follow these steps: Click the "Profile" option on the main menu. On the Profile screen, your profile information, such as username, first name, last name, and email address, will be displayed. The contexts state: "Editing Profile Step 1. Click Profile on the main menu. Step 2. Click the Edit button. Step 3. Edit the profile, including nick name, first name, and last name, as you want and click the Submit button. Step 4. A message "Change profile successful." is prompted, indicating the profile has been edited successfully. Click the Ok button." So in summary, to check your profile, you need to navigate to the "Profile" section on the main menu. Your current profile information will be displayed on this screen. Source: [Editing Profile] - mPOS Merchant Quick Start Guide

From above, you can see that "original question" and "alternative question" are the same but do not capture by knowledge database. How to resolve question like this?

[statefb]

Now the token limit is larger than 4096. We need to modify the frontend so that the app can handle large chunks.

[dezhouc2]

Now the token limit is larger than 4096. We need to modify the frontend so that the app can handle large chunks.

Thanks for the reply. I'm also wondering how knowledge database can resolve some issues like paraphrasing? For example, if I ask the same question in an alternative way, the response will be "Unfortunately, the contexts provided do not contain any information ... "

[statefb]

Now the app has limitation shown as the attached. If you need to paste all contexts to the chat, we need to modify the MAX value. Please note that this parameter is not related to RAG.

[dezhouc2]

Now the app has limitation shown as the attached. If you need to paste all contexts to the chat, we need to modify the MAX value. Please note that this parameter is not related to RAG.

Now the app has limitation shown as the attached. If you need to paste all contexts to the chat, we need to modify the MAX value. Please note that this parameter is not related to RAG.

Thanks for the reply. When can I see this? I'm still confused with how maximum token number is related to paraphrasing capability of knowledge database.

[k70suK3-k06a7ash1]

@dezhouc2 https://github.com/aws-samples/bedrock-claude-chat/pull/449/files

I have increased the max token.

[statefb]

@k70suK3-k06a7ash1 Thank you so much! @dezhouc2 I'd be appreciated if you could wait until the PR merged.

[statefb]

@dezhouc2 Sorry, I misunderstood. The concepts of context window and max tokens are different. The context window (currently 200K) pertains to the input to the LLM, while max tokens (the 4096 you mentioned) is a parameter related to the output. The context window is set automatically, so considering that the current context window size of 200K is sufficient for most issues, your problem likely arises from not being captured in the search results. Please try increasing the number of search results and try again. If it still doesn't work, you will need to check the cited search results in the CloudWatch logs.

[github-actions[bot]]

This issue has been labeled as "stale" due to no response by the reporter within 1 month (and 14 days after last commented by someone). And it will be closed automatically 14 days later if not responded.