RAI is a multi-vendor agent framework for robotics, utilizing Langchain and ROS 2 tools to perform complex actions, defined scenarios, free interface execution, log summaries, voice interaction and more.
Is your feature request related to a problem? Please describe.
RAI is typically successful with task plans such as go to A, ask about something, observe etc, go to B. However, it would be great for it to be able to support visualisation of tasks to help people understand what it is going to do. It makes most sense to visualise on a mobile device / website (webviz etc.).
Describe the solution you'd like
Produce visualization / markers for the plan and during it's execution:
In general, we need to be able to see the map as it understands it. This is existing feature to be included in rviz2 config or similar.
For a navigation task, nav stack waypoints and plan should be used. This is existing feature to be included in rviz2 config or similar.
(RAI addition) Task icons for tools that are used (say something, wait for response, observe), and a general task icon.
As a bonus, the tool could visualize keyframes from issue #213.
As a bonus, if it makes sense, try to make RAI additionally self-aware of this plan as a means of validation, by feeding the plan visually back to RAI (through a VLM description).
This could also be a part of RAI control UX in the future.
Is your feature request related to a problem? Please describe. RAI is typically successful with task plans such as go to A, ask about something, observe etc, go to B. However, it would be great for it to be able to support visualisation of tasks to help people understand what it is going to do. It makes most sense to visualise on a mobile device / website (webviz etc.).
Describe the solution you'd like Produce visualization / markers for the plan and during it's execution:
As a bonus, if it makes sense, try to make RAI additionally self-aware of this plan as a means of validation, by feeding the plan visually back to RAI (through a VLM description).
This could also be a part of RAI control UX in the future.