search

RobotecAI / rai

RAI is a multi-vendor agent framework for robotics, utilizing Langchain and ROS 2 tools to perform complex actions, defined scenarios, free interface execution, log summaries, voice interaction and more.
Apache License 2.0
80 stars 8 forks source link

Chat interface has no access to some tools when running HMI without the "whoami" parameter. #156

Open pijaro opened 3 weeks ago

pijaro commented 3 weeks ago

Describe the bug Running the HMI without robot description parameter:

streamlit run src/rai_hmi/rai_hmi/streamlit_hmi_node.py

has some limitations, ie. it can't fetch the camera output.

When specifying the robot description, for example

ros2 run rai_whoami rai_whoami_node --ros-args -p robot_description_package:="rosbot_xl_whoami"
streamlit run src/rai_hmi/rai_hmi/streamlit_hmi_node.py rosbot_xl_whoami

then everything is fine.

Is it expected behavior? If so, maybe some user feedback could be helpful. Also README should mention any limitations.

To Reproduce Steps to reproduce the behavior:

  1. Run demo simulation
  2. Run streamlit run src/rai_hmi/rai_hmi/streamlit_hmi_node.py
  3. Ask for camera output.
  4. Camera output is not displayed, the error mentions that the rclpy.Node object has no attribute 'get_raw_message_from_topic'

Expected behavior An image from the camera.

Platform

Version e0cff74cf4a0275a3301a3061442d60797297b57

maciejmajek commented 3 weeks ago

Is it expected behavior? If so, maybe some user feedback could be helpful. Also README should mention any limitations.

It is for now. We are currently working on HMI refactor. README will also be updated. Thank you!