kevinthedang / discord-ollama

Discord Bot that utilizes Ollama to interact with any Large Language Models to talk with users and allow them to host/create their own models.
Creative Commons Attribution 4.0 International
42 stars 4 forks source link

Infinite Message Length for Streaming #56

Closed kevinthedang closed 3 weeks ago

kevinthedang commented 2 months ago

Issue

Notes

Sample from messageNormal.ts

// run query based on stream preference, true = stream, false = block
if (stream) {
    response = await streamResponse(params)
    for await (const portion of response) {
        // append token to message
        result += portion.message.content

        // exceeds handled length
        if (result.length > 2000) {
            message.channel.send(`Response length ${result.length} has exceeded Discord maximum.\n\nLong Stream messages not supported.`)
            break // stop stream
        }

        // resent current output, THIS WILL BE SLOW due to discord limits!
        sentMessage.edit(result || 'No Content Yet...')
    }
} else { ... }