search

fujitatomoya / ros2ai

ros2ai is a next-generation ROS 2 command line interface extension with AI such as OpenAI
https://github.com/fujitatomoya/ros2ai
Apache License 2.0
147 stars 10 forks source link

[FR] support session mode to keep the context during session #34

Open fujitatomoya opened 2 months ago

fujitatomoya commented 2 months ago

Currently ros2ai uses only single chat completion, that means ros2ai does not have any idea about what questions or queries are asked by user previously. ros2ai starts the new chat completion request every time via command line. This is a hard constraint for user experience, because that is the user's responsibility to put the previous context in the current request description every time.

this actually disables users to do like,

  1. Do we have topic name /chatter?
  2. Can you subscribe it?

this it cannot be recognized by ros2ai currently.

Since this is really general use case and user would expect this to work with ros2ai, this needs to be supported.

atm, i am not sure if OpenAI Python API supports this with API, but if it does, ros2ai can just relies on that. But if not supported, once the session mode is enabled, ros2ai can cache the data and descriptions in previous, and add it in the current chat completion every time. the latter case, it would increase the token consumption as completion history increases, but there would be nothing else we really can do.

fujitatomoya commented 2 months ago

according to,

i do not think general API to support this history or session in openai/openai-python side. that means ros2ai needs to save the history and append those to the current request via completion request every time. there would be limitation of tokens for the completion API, if we hit this limit, the oldest history needs to be removed.