Agenta-AI / agenta

The all-in-one LLM developer platform: prompt management, evaluation, human feedback, and deployment all in one place.
http://www.agenta.ai
MIT License
1.14k stars 177 forks source link

[AGE-290] [Bug] MultipleChoiceParam not shown in UI #1747

Closed shhhhhh2001 closed 3 months ago

shhhhhh2001 commented 3 months ago

Describe the bug When creating a custom APP, after adding MultipleChoiceParam in ag.config.register_default(), nothing shows in the UI.

To Reproduce Steps to reproduce the behavior:

  1. Add ag.config.register_default( ... model=ag.MultipleChoiceParam("...", llm_list) ) in python
  2. "agenta variant serve py file" in cli
  3. Open the playground in UI
  4. See error

Expected behavior A dropdown menu should appear.

Desktop (please complete the following information):

AGE-290

dosubot[bot] commented 3 months ago

Hello @shhhhhh2001! I'm here to help you with any bugs, questions, or contributions. Let's work together to resolve the issue you're experiencing.

The MultipleChoiceParam is not showing in the UI because it might not be registered correctly. Ensure that you are using the ag.config.register_default() method properly.

Here is the corrected code:

import agenta as ag
from langchain.llms import OpenAI
from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate

default_prompt = "What is a good name for a company that makes {product}?"

ag.init()
ag.config.register_default(
    prompt_template=ag.TextParam(default_prompt),
    model=ag.MultipleChoiceParam(default="1", choices=["1", "2"]),
)

@ag.entrypoint
def completion(
    product: str,
) -> str:
    llm = OpenAI(model=ag.config.model)
    prompt = PromptTemplate(
        input_variables=["product"],
        template=ag.config.prompt_template,
    )
    chain = LLMChain(llm=llm, prompt=prompt)
    output = chain.run(product=product)

    return output

By using ag.config.register_default() and ensuring the MultipleChoiceParam is correctly instantiated with a default value and choices, the parameter should now be properly registered and displayed in the UI [1][2].

To continue talking to Dosu, mention @dosu.

mmabrouk commented 3 months ago

Thank you @shhhhhh2001

We will try to reproduce it and fix it in case we are able to. Can you please provide the exact snippet that you have used.

In the mean time, I recommend using GroupedMultipleChoice. Here is an example:

supported_llm_models = {
    "Mistral AI": [
        "mistral/mistral-tiny",
        "mistral/mistral-small",
        "mistral/mistral-medium",
        "mistral/mistral-large-latest",
    ],
    "Open AI": [
        "gpt-3.5-turbo-16k-0613",
        "gpt-3.5-turbo-16k",
        "gpt-3.5-turbo-1106",
        "gpt-3.5-turbo-0613",
        "gpt-3.5-turbo-0301",
        "gpt-3.5-turbo",
        "gpt-4",
        "gpt-4-1106-preview",
    ],
}
model=ag.GroupedMultipleChoiceParam(default="gpt-3.5-turbo", choices=supported_llm_models)
mmabrouk commented 3 months ago

@ezealigokosiso can you please attempt to reproduce the bug.

shhhhhh2001 commented 3 months ago

Thank you @shhhhhh2001

We will try to reproduce it and fix it in case we are able to. Can you please provide the exact snippet that you have used.

In the mean time, I recommend using GroupedMultipleChoice. Here is an example:

supported_llm_models = {
    "Mistral AI": [
        "mistral/mistral-tiny",
        "mistral/mistral-small",
        "mistral/mistral-medium",
        "mistral/mistral-large-latest",
    ],
    "Open AI": [
        "gpt-3.5-turbo-16k-0613",
        "gpt-3.5-turbo-16k",
        "gpt-3.5-turbo-1106",
        "gpt-3.5-turbo-0613",
        "gpt-3.5-turbo-0301",
        "gpt-3.5-turbo",
        "gpt-4",
        "gpt-4-1106-preview",
    ],
}
model=ag.GroupedMultipleChoiceParam(default="gpt-3.5-turbo", choices=supported_llm_models)

Thank you for your suggestion. But GroupedMultipleChoiceParam is also not working...

mmabrouk commented 3 months ago

Hi @shhhhhh2001 we were not able to reproduce the bug. Can you please make sure to use the latest version of the CLI pip install -U agenta and in the case you are self-hosting pull and migrate to the latest version of agenta.

shhhhhh2001 commented 3 months ago

Is it possible to show my codes to someone in some way? I follow all the instructions you provided, use the latest version of CLI, and upload my app on your cloud platform. But still nothing appears.

mmabrouk commented 3 months ago

I was able to reproduce the issue and understand the source of the problem. GroupedMultipleChoice is not rendered correctly if there is no FloatParam in the configuration. I am preparing a fix to the issue

For now, here is a hotfix, please modify the configuration to add a FloatParam (even if you don't plan to use it):

ag.config.default(
    temperature=ag.FloatParam(default=1, minval=0.0, maxval=2.0),  # this should be added
    model=ag.GroupedMultipleChoiceParam(
        default="miao", choices=supported_llm_models
    ),
    prompt_system=ag.TextParam(system_prompt),
    prompt_user=ag.TextParam(user_prompt),
)