search

vectara / create-ui

Generate a working React codebase for a range of generative and semantic search UIs
https://vectara.github.io/create-ui
Apache License 2.0
69 stars 12 forks source link

Make a Vectara Create-UI chatbot use the MockingBird LLM #40

Open hello007world opened 3 months ago

hello007world commented 3 months ago

I’m attempting to use the MockingBird model to make a chatbot only answer questions related to the .PDF document data in the corpus (so a question like “How is the weather?” or “How are you?” should not give out a long answer but rather a simple “Please try a different search or restate your question in another way” message)

When creating a chatbot using the ‘npx @vectara/create-ui’ command, there is no option to set the summary model to anything other than ‘vectara-summary-ext-v1.2.0’, as far as I have seen.

Trying to change it manually in ‘SearchContext.tsx’ to ‘mockingbird-1.0-2024-07-16’ yields no changes in the summarization.

I’ve tried to see the docs, API docs, and this page for any possible way to change the setup of the summary model but haven’t found any success.

If it helps, I used this tutorial to create the chatbot itself: https://www.youtube.com/watch?v=HoBuGPhi6Zs

If anyone has a solution, I would appreciate your help.

cjcenizal commented 3 months ago

Thanks for raising this @hello007world! I just tried to reproduce this by updating https://github.com/vectara/create-ui/blob/main/apps/chat/src/contexts/SearchContext.tsx#L203 with a generation config for Mockingbird:

const streamQueryConfig: ApiV2.StreamQueryConfig = {
  generation: {
    promptName: "mockingbird-1.0-2024-07-16b"
  },
  /* snip */
};

Then I tried out the Chat app to see if Mockingbird responded correctly to a nonsensical question, and it behaved as I hoped it would:

image

Does this help? If not could you copy/paste your streamQueryConfig object so I can try to debug?

hello007world commented 3 months ago

@cjcenizal Thank you so much for the solution!

Funny thing is that it actually worked better using 'vectara-summary-ext-24-05-sml', as MockingBird would answer a question like "what is the weather" while the other model wouldn't.

Images to show what I mean (note that the MockingBird snapshot had some portions blanked out): chatbot_weather_mockingbird_ chatbot_weather_ext24-05-sml