Add API flag `retrieval_only` - retrieving ONLY contexts, no LLM invoked

[rohan-uiuc]

Contexts are returned by default on non-streaming API. Added flag retrieval_only

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
chat-illinois-edu	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 8, 2024 8:43pm
uiuc-chat	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 8, 2024 8:43pm

[rohan-uiuc]

Yep, docs is next, starting that. RE:it would be great to add this to the non-streaming API by default. It is added to the non streaming API by default in the last commit.

[KastanDay]

Ok I modified the examples on /api

• Default to llama3.1 so that the 'copy and paste' works without modification.
• Emphasize in writing that GPT-4o-mini is the best price/perf model.

[KastanDay]

Added a link to our docs page, too:

[rohan-uiuc]

The model we host can change any time, we will have to modify this everytime that happens. Hence, 4o-mini in the example request. "How to use NCSA hosted models" is already present in the documentation. Additionally, the performance with 4o-mini is better so I would prefer that as a default. The model mentions llama but there is a field for openai_key in the default example which will only confuse the user. All the various combinations were kept in the documentation to keep default request simple.

[KastanDay]

Ok that's a convincing argument. I agree with gpt-4o-mini perf being so much better that we should keep it default.

[rohan-uiuc]

This information is already available in the documentation.

[KastanDay]

This information is already available in the documentation.

Agree, but I still think it's worth a callout here. So I'm voting to keep it for now.

[rohan-uiuc]

This information is already available in the documentation.

Agree, but I still think it's worth a callout here. So I'm voting to keep it for now.

Can we move this to a separate PR, do a review of the UI and then push this? I believe this PR is ready to merge without these changes. The new changes donot match fonts styles, increase the text by a lot on this page and are not related to the fix at all, the weird black color is hurting my eyes.

[KastanDay]

I like the black code formatting. I could make the contrast less intense if you insist. But we use it elsewhere.

Let's just merge it as is. The styling isn't bad.