lmarena / arena-hard-auto

Arena-Hard-Auto: An automatic LLM benchmark.
Apache License 2.0
604 stars 71 forks source link

QA browser does not work properly for me #2

Closed suquark closed 6 months ago

suquark commented 6 months ago

Environment

python3.10, with gardio==4.26.0, gradio_client==0.15.1

Typo

Well, first there is a typo in the file name. It should be "browser" instead pf "broswer".

How to reproduce it

First, there are no dependencies of gradio in requirements. So first I have to manually install gradio to run the qa browser

After invoking python qa_broswer.py, there is something abnormal:

Namespace(host='0.0.0.0', port=None, share=False, config_file='config/judge_config.yaml')
<...>/python3.10/site-packages/gradio/components/dropdown.py:179: UserWarning: The value passed into gr.Dropdown() is not in the list of choices. Please update the list of choices to include: gpt-3.5-turbo-0613 or set allow_custom_value=True.
  warnings.warn(
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.

When accessing the link,

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  <skip some not very relevant messages>
  File "xxx/python3.10/site-packages/gradio/routes.py", line 797, in process_msg
    return f"data: {orjson.dumps(message.model_dump()).decode('utf-8')}\n\n"
TypeError: Type is not JSON serializable: Dropdown

And this is what I observed

image
CodingWithTim commented 6 months ago

Thanks Siyuan. I fixed the issue and pushed new code. It seems like gradio==3.40.0 and need httpx==0.25.2 are required. Let me know if this fixes for you and if not let me know.

suquark commented 6 months ago

Now it is working smoothly! Thank you a lot.

BTW, could you also add these extra dependencies somewhere (README or requirements-optional.txt)? It is pretty hard to figure out what is the proper version of gardio

CodingWithTim commented 6 months ago

Yes that is a great idea. Thanks Siyuan.