How to use ChatGPT API to continue a conversation without clearing the previous API call context

[surendransuri]

I'm trying to use the ChatGPT API to continue a conversation with a chatbot. However, I noticed that each API call clears the previous context, making it difficult to maintain the conversation flow. Can someone please help me understand how to use the ChatGPT API to continue a conversation without clearing the previous context? I'm looking for guidance on the API parameters or coding techniques that I can use to preserve the conversation context across multiple API calls. Any examples or code snippets would be greatly appreciated.

Additional information: I have tried passing the context parameter in the API call to preserve the conversation context, but it does not seem to work. I am using the OpenAI Python library to make API calls.

[FrackinFamous]

You have to either create embeddings of the chat history and save those in a vector database like pinecone or append the messages to the previous one. There is currently no way to "save" this info through the API that I know of.

I think this is what you are looking for and should get you started down the path. You don't have to use LangChain but it's a good place to start to get the ideas going. Let me know if this helps!

https://python.langchain.com/en/latest/modules/memory/getting_started.html

[OrlyVieraH]

@surendransuri To build the chat structure, you just need to create a new API request using the previous request and completion. It's an easy way to do it, but remember to keep in mind the token limits for the API you're using.

[ted-at-openai]

There's no memory. You have to pass in the chat history in each API call.

[BA2255]

Is there a solution to this problem? How is the POE app doing this, I don't think every time they are sending the whole chat history to ChatGPT, otherwise, it would become very expensive.

[ted-at-openai]

You have to pass in the chat history in each API call. You can potentially play tricks with which parts of the chat history to retrieve/discard, etc.

[BA2255]

So everyone using Open AI APIs is sending the whole chat again, this is then using a lot of resources. What about the website version of Chat GPT, does do that too? Wouldn't there be a way to tag each API, and to run a temporary memory or cache inside the model? Not sure if this is technically possible, but this would save Open AI and the customers significant processing.

[jadaliha]

So everyone using Open AI APIs is sending the whole chat again, this is then using a lot of resources. What about the website version of Chat GPT, does do that too? Wouldn't there be a way to tag each API, and to run a temporary memory or cache inside the model? Not sure if this is technically possible, but this would save Open AI and the customers significant processing.

OpenAI chat webpage is using the parameter "parent_message_id". Unfortunately, I could not properly pass this parameter to the API endpoints. The possibility is there, however, you may not be able to use it from the API endpoint that you and I have access to.

[dearsina]

Check out this article. There is a simple fix that uses a naive session approach to handle this https://betterprogramming.pub/a-simple-technique-for-preventing-conversation-drift-in-llm-based-chatbots-f896756adecb

all you have to do is change the session id and the chat history will be cleared.

This isn't a novel approach, it's doing what OP is trying to avoid doing, but simply appending the previous conversation to the new request. This becomes very expensive.

[MrVibe]

I want to upvote this feature request. It becomes really expensive to re-share the entire chat and responses.

[BA2255]

I think this is what PI.ai does, it summarises chats, otherwise as you see it becomes super expensive to send the whole chat everytime there is an interaction.

[janfilips]

Yeah that would quite useful

[Kalmuraee]

That would help, a temp cache to identify the conversation with user generated chatID

[echan00]

did anyone find a solution to this?

[dnc-nl]

Hello all,

So, I'm creating an internal tool that allows us to check for bugs, and do internal technical documentation of our c#/angular projects. I think it's really important that the conversation understands the full context of what it is working with to see the big picture. With Chat GPT you could provide all the different cs files and thus create one big conversation with the full context.

How do you think you can work around this 'limitation' I'm looking at sementic kernel, but I'm not sure this will do the trick, if you start summarizing code it doesn't really make sense, or am I missing something here? Has anyone have an idea how to solve this core issue?

[stcalica]

Instead of passing in the entire chat history, you want to discuss a topic that it has specific knowledge about in a context of your choice. I'm legitimately asking for my own purposes, but a better solution is to train it on data around the topic, no?

[alexzender]

You need to pass all previous messages via Completion API, and if the text becomes too large to pass everything, you can do an API call to summarize your previous chat history, and so reduce the context

[jpeg2600]

Seems like you can now create assistants and threads to maintain a conversation, you just have to send a thread ID instead of the whole thing to keep things going

Neat feature. I'm struggling to get a JSON every time but that's another matter.

[zgordan-vv]

Seems like you can now create assistants and threads to maintain a conversation, you just have to send a thread ID instead of the whole thing to keep things going

Neat feature. I'm struggling to get a JSON every time but that's another matter.

Where to find this thread id? I got only something like

role: 'assistant',
id: 'chatcmpl-8k7A8chCgmtgCSFXWqtT5Bl3lJtdB',
conversationId: 'chatcmpl-8k6wrMOaCxLHKIZseSBDfHwkUSt4r',
parentMessageId: 'aa7249ea-1433-4cbc-ba93-4ef608c1eacd
detail: {
    id: 'chatcmpl-8k7A8chCgmtgCSFXWqtT5Bl3lJtdB',
    object: 'chat.completion',
    created: 1706001312,
    model: 'gpt-3.5-turbo-0613',
    choices: [ [Object] ],
    usage: { prompt_tokens: 93, completion_tokens: 322, total_tokens: 415 },
    system_fingerprint: null
}

but nothing of it can help to continue the same conversation

[zhuyinheng]

@zgordan-vv doc: https://platform.openai.com/docs/assistants/overview demo: https://github.com/openai/openai-python/blob/main/examples/assistant.py