How can I use the Response streaming technique in a POST method?

[peilongchencc]

@ahopkins

I apologize for asking about the issue #2730 you've already marked as "Closed as not planned," but I really need your assistance.

Hello, I would like to fetch user's historical chat data through the post method and then pass this chat data to a large language model's API.

The large language model returns content via streaming output. I will then process the results returned by the large language model and subsequently stream out the processed results.

My code structure is as follows:

from sanic import Sanic
from sanic import response

app = Sanic("my_app")

@app.route("/ans", methods=["POST"])
async def answer(request):
    # fetch user's historical chat data.
    chat_history = request.form.get("chat_history")

    url = "https://xxx/sse/paas4Json/..."
    header = {
        'Accept': 'text/event-stream'
    }
    # Invoke the API of a large language model.
    response = requests.post(url=url, data=chat_history, stream=True, headers=header)
    response.raise_for_status()

    # Parse the streaming output results of a large language model.
    for line in response.iter_lines():
        line = line.decode('utf-8')
        if line and line != '' and line.startswith('data:'):
            data_content_dict = json.loads(line)
            value = json.loads(data_content_dict["data"][0]["value"])
            rtn = value["MessageBody"]["DirectMessageBody"]["SentenceList"][0]["Content"]
            # I would like to return `rtn` in a streaming manner.
            yield rtn

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8848)

I would like to return rtn in a streaming manner. I referred to the code on your website (https://gist.github.com/ahopkins/9cdf98147da863a0c1c5b0b8581d6980), but due to my limited understanding of web communications, I'm not sure how to use the Response streaming technique within the post method.

Could you help me with an example of a post method that uses Response streaming?

I hope to get your assistance with this.

Ps: My Sanic version is 23.12.1.

[Tronic]

Refer to this for response streaming: https://sanic.dev/en/guide/advanced/streaming.html#response-streaming

Closing. Please address support questions to Sanic community forums or to our Discord #support group.

[ahopkins]

See this for help. https://sanic.dev/en/help.html

Community support is in Discord or our forum.