Request: more clarity on JSON response from custom models

[thubreg]

Note: duplicate from https://huggingface.co/spaces/huggingchat/chat-ui/discussions/309, not sure which is the proper place to post.

I followed the guide chat-ui to deploy a version in gcp, and I love the chat interface.

I would love to hook it up to one of my custom models, so I specified

"endpoints": [{"url": "[http://127.0.0.1:8000"}]](http://127.0.0.1:8000"%7D%5D/)
}
]`

for MODELS as suggested.

I receive the message that has been posted in the web interface at my endpoint, but I am unable to send back the proper json response. So far, in python, I do:

response_content = [
{
"generated_text": "Please show this response."
}
]
response = make_response(jsonify(response_content))
return response

It is received in the chat-ui code (confirmed by injecting console.log statements), but it doesn't show in the browser conversation.

Can someone please clarify what json (content, headers, whatever is needed) I need to send from my custom model endpoint as a response to the chat-ui interface? Or if this is the wrong place to ask, tell me where I should ask?

[julien-blanchon]

Hey @thubreg, you can have a look to https://github.com/gururise/openai_text_generation_inference_server/blob/main/server.py. This might help you understand how to format your endpoint