Strvm / meta-ai-api

Llama 3 API 70B & 405B (MetaAI Reverse Engineered)
267 stars 50 forks source link

Stream only the new part #38

Closed tqdo closed 1 month ago

tqdo commented 1 month ago

When streaming the response, is there any way to only stream the new part (the part that is not the previous response yet) only? Right now in the streaming example, the response is cumulative, meaning the new response includes both the previous response and the new part. Thanks