let user select LLM that generates final response

MaxPowerWasTaken commented 3 days ago

tasks:

add more API Keys to frontend secrets file (local & on streamlit community cloud for deployment)
replace some google-generative-ai sdk code with litellm to simplify code for switching between LLM models
add dropdown selection menu for LLM to frontend. should default to whatever's in config.py
restructure config.py so that it separates between user-adjustable config params, and ones that aren't

MaxPowerWasTaken commented 3 days ago

claude-3-5-sonnet-20241022 throws 'bad request error': "Your credit balance is too low to access the Anthropic API. Please go to Plans & Billing to upgrade or purchase credits."

same with claude-3-5-haiku-20241022.

command-r-plus works (and gives an interestingly different answer; more extensive and bulleted list)

obviously the gemini ones still work, with their crazy free tier.

GPTs: NotFoundError: litellm.NotFoundError: OpenAIException - Error code: 404 - {'error': {'message': 'The modelgpt-3.5-turbo-16k-0613has been deprecated,...

gpt-4-turbo-2024-04-09 works

chatgpt-4o-latest works and gives my favorite answer so far. extensive structured list of reasons like command-r-plus, but instead of bullet points it's a numbered list with a bold title/topic, then explanatory text for.

gpt-4o-mini works.

o1-mini-2024-09-12 works and gives a similarly impressive answer to 4o-latest, same format, pretty cool for a 'mini' model.

UPDATE: ok cool for a 'mini' model I guess, but cost of o1-mini is $3.00 / 1M input tokens & $12.00/1M output tokens, slightly more than $2.50/1M input tokens & $10.00/1M output tokens for gpt-4o & gpt-4o-2024-11-20 per https://openai.com/api/pricing/ (And curiously, though you'd think the 4o's listed above are the latest, that pricing page lists chatgpt-4o-latest separately as more expensive than them; $5 & $15 per 1M input & output tokens...

CONCLUSION: let's remove claude/anthropic options for their apparently stingier free tier (or keep it but set up a spending limit on my api key), and remove the deprecated gpt-turbo option... ...and let's change default option to o1-mini.

MaxPowerWasTaken commented 3 days ago

....interesting note from openai doc on deprecations:

On November 6th, 2023, we [announced](https://openai.com/blog/new-models-and-developer-products-announced-at-devday) the release of an updated GPT-3.5-Turbo model (which now comes by default with 16k context) along with deprecation of gpt-3.5-turbo-0613 and gpt-3.5-turbo-16k-0613. As of June 17, 2024, only existing users of these models will be able to continue using them.

...so it seems that gpt-3.5-turbo-16k-0613 is still a valid model for some litellm users, just not me. so probably should stay in litellm.utils.get_valid_models(), just not in my app.

MaxPowerWasTaken commented 2 days ago

didn't want to remove claude models altogether just because of no free tier. still very cheap unless usage suddenly gets very heavy. added $10 in API credits to my claude key and set auto-refill but with $20 monthly spend limit.

MaxPowerWasTaken commented 2 days ago

closed by https://github.com/MaxPowerWasTaken/askliz_frontend/commit/70863e932054f10e27acb86ffebbb291c6e026ae

MaxPowerWasTaken / askliz_frontend

let user select LLM that generates final response #5