Open d3287t328 opened 1 year ago
Any ideas. This is great but without claude the token issue prevents from using it. Tried chunking and other options but no luck.
Same here I'm assuming the context size from Claude is the defining factor.
openai.error.InvalidRequestError: This model's maximum context length is 8192 tokens. However, your messages resulted in 9037 tokens. Please reduce the length of the messages.
Seen some videos with people having some really detailed prompt.md files.
yeah unfortunately I cant do much about claude access :/
Hi @jdingus @d3287t328 @BigwigsNFT I believe I can help with this issue. I’m the maintainer of LiteLLM https://github.com/BerriAI/litellm - we allow you to use any LLM as a drop in replacement for gpt-3.5-turbo
.
You can use LiteLLM in the following ways:
This calls the provider API directly
from litellm import completion
import os
## set ENV variables
os.environ["OPENAI_API_KEY"] = "your-key" #
os.environ["COHERE_API_KEY"] = "your-key" #
messages = [{ "content": "Hello, how are you?","role": "user"}]
# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)
# cohere call
response = completion(model="command-nightly", messages=messages)
this is great if you don’t have access to claude but want to use the open source LiteLLM proxy to access claude
from litellm import completion
import os
## set ENV variables
os.environ["OPENAI_API_KEY"] = "sk-litellm-5b46387675a944d2" # [OPTIONAL] replace with your openai key
os.environ["COHERE_API_KEY"] = "sk-litellm-5b46387675a944d2" # [OPTIONAL] replace with your cohere key
messages = [{ "content": "Hello, how are you?","role": "user"}]
# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)
# cohere call
response = completion(model="command-nightly", messages=messages)
What do you suggest for those of us without a clause api key to use this? I applied for one but it doesn't seem like it is going to be provided to me anytime soon.