Implement assistant API streaming

mme commented 7 months ago

Is your feature request related to a problem? Please describe. OpenAI just launched streaming support in the assistant API. CopilotKit should support streaming responses via the assistant API.

Describe the solution you'd like Modify https://github.com/CopilotKit/CopilotKit/blob/main/CopilotKit/packages/backend/src/lib/openai-assistant-adapter.ts to use streaming. See: https://platform.openai.com/docs/assistants/overview?context=with-streaming

CopilotKit commented 7 months ago

/bounty $15

algora-pbc commented 7 months ago

💎 $15 bounty • CopilotKit

Steps to solve:

Start working: Comment /attempt #266 with your implementation plan
Submit work: Create a pull request including /claim #266 in the PR body to claim the bounty
Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to CopilotKit/CopilotKit!

Add a bounty • Share on socials

Attempt	Started (GMT+0)	Solution
🟢 @pi600	Mar 14, 2024, 12:01:11 PM	WIP
🔴 @chintanpuggalok	Apr 5, 2024, 6:53:07 PM	WIP
🟢 @abhijitxy	May 3, 2024, 7:24:56 AM	WIP
🔴 @hrshkshri	May 8, 2024, 6:26:59 AM	WIP
🟢 @amaanlari	May 8, 2024, 8:50:32 PM	WIP

Bhavyajain21 commented 7 months ago

Can I be assigned?

pi600 commented 7 months ago

/attempt #266

ataibarkai commented 7 months ago

@Bhavyajain21 yes.

Bhavyajain21 commented 7 months ago

Please assign

ataibarkai commented 7 months ago

@Bhavyajain21 it's assigned

Bhavyajain21 commented 7 months ago

@ataibarkai Thanks for assigning

Bhavyajain21 commented 7 months ago

@mme / @ataibarkai
The provided code under (https://github.com/CopilotKit/CopilotKit/blob/main/CopilotKit/packages/backend/src/lib/openai-assistant-adapter.ts) already utilizes streaming to communicate with the OpenAI API for assistant interactions.

The provided documentation mentions streaming support but doesn't go into specific implementation details. It emphasizes the ability to continue conversations through threads, which aligns with the current code's approach.

Is there anything different we are expecting out of it?

mme commented 7 months ago

Hi @Bhavyajain21 ! According to the OpenAI docs, there should be a new method available called createAndStream. Can you have a look at this method? You would need to update the openai package to the lastest version.

Bhavyajain21 commented 7 months ago

Got that, thanks for the input!