Restrict to only GPT-3.5?

[Itangalo]

Thanks for this awesome project.

I've looked at settings and environment variables (and also some code), but can't find a way to restrict Chatbot UI to only use gpt-3.5-turbo (not GPT-4). I'm planning on having a publically available chatbot for my organization, and limiting to gpt-3.5-turbo would make it fit better within our budget.

I'm not sure whether this is a support request or a feature request, but I'm adding it anyway.

[robertdalla]

All you need to do is to comment-out the gpt-4 references in file /types/openai.ts See below:

`import { OPENAI_API_TYPE } from '../utils/app/const';

export interface OpenAIModel { id: string; name: string; maxLength: number; // maximum length of a message tokenLimit: number; }

export enum OpenAIModelID { GPT_3_5 = 'gpt-3.5-turbo', GPT_3_5_AZ = 'gpt-35-turbo', // GPT_4 = 'gpt-4', // GPT_4_32K = 'gpt-4-32k', }

// in case the DEFAULT_MODEL environment variable is not set or set to an unsupported model export const fallbackModelID = OpenAIModelID.GPT_3_5;

export const OpenAIModels: Record<OpenAIModelID, OpenAIModel> = {

id: OpenAIModelID.GPT_3_5,
name: 'GPT-3.5',
maxLength: 12000,
tokenLimit: 4000,

},

id: OpenAIModelID.GPT_3_5_AZ,
name: 'GPT-3.5',
maxLength: 12000,
tokenLimit: 4000,

}, // [OpenAIModelID.GPT_4]: { // id: OpenAIModelID.GPT_4, // name: 'GPT-4', // maxLength: 24000, // tokenLimit: 8000, // }, // [OpenAIModelID.GPT_4_32K]: { // id: OpenAIModelID.GPT_4_32K, // name: 'GPT-4-32K', // maxLength: 96000, // tokenLimit: 32000, // }, };`

[Itangalo]

Many thanks! (bows)

[stigmartins1]

I have the opposite problem - only 1 model is available in the drop-down menu for models in the UI. I just cloned the repo yesterday and haven't commented out anything.

[robertdalla]

It seems that the repo is not maintained anymore :( Therefore the latest available models are not included (yet). As per 14 July 2023 the gpt models available from openAI are:

gpt-3.5-turbo gpt-3.5-turbo-16k gpt-4 gpt-4-32k

And below is code that includes the above models (file /types/openai.ts)

import { OPENAI_API_TYPE } from '../utils/app/const';

export interface OpenAIModel { id: string; name: string; maxLength: number; // maximum length of a message tokenLimit: number; }

export enum OpenAIModelID { GPT_3_5 = 'gpt-3.5-turbo', GPT_3_5_16K = 'gpt-3.5-turbo-16k', GPT_3_5_AZ = 'gpt-35-turbo', GPT_3_5_16K_AZ = 'gpt-35-turbo-16k', GPT_4 = 'gpt-4', GPT_4_32K = 'gpt-4-32k', GPT_4_AZ = 'gpt-4', GPT_4_32K_AZ = 'gpt-4-32k', }

// in case the DEFAULT_MODEL environment variable is not set or set to an unsupported model export const fallbackModelID = OpenAIModelID.GPT_3_5;

export const OpenAIModels: Record<OpenAIModelID, OpenAIModel> = {

id: OpenAIModelID.GPT_3_5,
name: 'GPT-3.5',
maxLength: 12000,
tokenLimit: 4096,

},

id: OpenAIModelID.GPT_3_5_16K,
name: 'GPT-3.5-16k',
maxLength: 48000,
tokenLimit: 16384,

},

id: OpenAIModelID.GPT_3_5_AZ,
name: 'GPT-3.5',
maxLength: 12000,
tokenLimit: 4096,

},

id: OpenAIModelID.GPT_3_5_16K_AZ,
name: 'GPT-3.5-16k',
maxLength: 48000,
tokenLimit: 16384,

},

id: OpenAIModelID.GPT_4,
name: 'GPT-4',
maxLength: 24000,
tokenLimit: 8192,

},

id: OpenAIModelID.GPT_4_32K,
name: 'GPT-4-32K',
maxLength: 96000,
tokenLimit: 32768,

},

id: OpenAIModelID.GPT_4_AZ,
name: 'GPT-4',
maxLength: 24000,
tokenLimit: 8192,

},

id: OpenAIModelID.GPT_4_32K_AZ,
name: 'GPT-4-32K',
maxLength: 96000,
tokenLimit: 32768,

}, };

You can set a default model in a .env.local file (see also exemple in file .env.local.example) with DEFAULT_MODEL The exact value should match with one of the entry in above object OpenAIModelID otherwise the app will fallback and use model 'gpt-3.5-turbo' (which is displayed as ''GPT-3.5" in the UI).

.env.local file example:

DEFAULT_MODEL = gpt-3.5-turbo-16k NEXT_PUBLIC_DEFAULT_SYSTEM_PROMPT = You are ChatGPT 2023, a Large Language Model trained by OpenAI. Follow the user's instructions carefully. Respond using markdown. NEXT_PUBLIC_DEFAULT_TEMPERATURE = 1 OPENAI_API_HOST = https://api.openai.com OPENAI_API_TYPE = openai OPENAI_API_KEY = your API key OPENAI_ORGANIZATION = your organization ID

And lastly, gpt-4 and gpt-4-32k will only be available IF YOU HAVE UPGRADED TO A PREMIUM account, and sometime depending where you are based, you would still need to wait if they told you that you are on a waiting list. So regarding gpt-4 availability with Chatbot-UI you should first check from the openAI playground if you see gpt-4 in the Model dropdown widget. https://platform.openai.com/playground

[ChaoticG]

Hi, I have the 4.0 API, and I followed your method to modify the code in the GitHub repository. Now the web page correctly displays the GPT-4 model, and I've successfully engaged in conversation with it. However, it keeps insisting that it's GPT-3, and it can't answer the current time. Am I missing some sort of setting? @robertdalla

[ishaan-jaff]

@Itangalo You can use LiteLLM to fix your issue: https://github.com/BerriAI/litellm

LiteLLM - allows you to use any LLM as a drop in replacement fot gpt-3.5-turbo + You can set a $ budget per user or per session

from litellm import BudgetManager, completion 
budget_manager = BudgetManager(project_name="test_project")
user = "1234"

# create a budget if new user user
if not budget_manager.is_valid_user(user):
    budget_manager.create_budget(total_budget=10, user=user)

# check if a given call can be made
if budget_manager.get_current_cost(user=user) <= budget_manager.get_total_budget(user):
    # call gpt-3.5
    response = completion(model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hey, how's it going?"}])
    budget_manager.update_cost(completion_obj=response, user=user)
else:
    response = "Sorry - no budget!"