personalrobotics / ada_feeding

Robot-assisted feeding demos and projects for the ADA robot
4 stars 4 forks source link

[ROS2] AcquireFood should account for robot motions since getting the mask #126

Closed amalnanavati closed 8 months ago

amalnanavati commented 10 months ago

Currently, the AcquireFood action assumes the camera is in the same pose as it was when the mask was captured from the web app. Although that is mostly true, the one case where it might not be is if the app user refreshes the page while the robot is moving. As the page is unmounting, the app will terminate the action, and then as it is re-mounting, it will call the action with the same parameters as when it was originally called. That is because the React app cannot differentiate between the page mounting when the user just transitions to it, versus the page mounting when it refreshes, and the "robot motion" pages are designed to call the robot action as soon as they mount.

To address this, when AcquireFood computes the food frame, it should first use the TF tree to get the transform from the camera to the base frame at the time included in the action goal's header, and then compute the food frame relative to that.

(Note, there is a separate question of whether it is desirable to have the app re-call the robot action if the page is refreshed, and, if not, how we can architect the app to avoid that. However, the above seems like a good improvement to make to acquisition code regardless, and we can also make the change to app architecture if needed.)

amalnanavati commented 9 months ago

BTW @egordon friendly reminder about this. As you are changing to the new Mask type that now returns an RGB image, this might be an easy change to make.