Steps to use FastChat t5 as an API or GUI

[kush-2418]

1. terminal 1 - python3.10 -m fastchat.serve.controller --host localhost --port PORT_N1

2. terminal 2 - CUDA_VISIBLE_DEVICES=0 python3.10 -m fastchat.serve.model_worker --model-path <path_to_fastchat_model>/lmsys_fastchat-t5-3b-v1.0/ --controller-address http://localhost:PORT_N1 ## controller address should be same as given in terminal 1

3. if GUI _ terminal 3 - python3.10 -m fastchat.serve.gradio_web_server --host localhost --port PORT_N2 --controller-url http://localhost --port PORT_N1 ## controller address should be same as given in terminal 1

4. if API _ terminal 3 - python3.10 -m fastchat.serve.openai_api_server --host localhost --port PORT_N2 --controller-address http://localhost --port PORT_N1 ## controller address should be same as given in terminal 1

Now change the address in both the files to http://localhost --port PORT_N2 and in another terminal run python3.10 test_openai_sdk.py or bash test_openai_curl.sh

NOTE -preferably use python3.10 version -instead of localhost you can also mention your server address

If KeyError: 'embedding' then delete the test_embedding() function from test_open_ai_sdk.py file

[merrymercy]

The embedding api will be fixed by #1246