Currently, the AcquireFood action assumes the camera is in the same pose as it was when the mask was captured from the web app. Although that is mostly true, the one case where it might not be is if the app user refreshes the page while the robot is moving. As the page is unmounting, the app will terminate the action, and then as it is re-mounting, it will call the action with the same parameters as when it was originally called. That is because the React app cannot differentiate between the page mounting when the user just transitions to it, versus the page mounting when it refreshes, and the "robot motion" pages are designed to call the robot action as soon as they mount.
To address this, when AcquireFood computes the food frame, it should first use the TF tree to get the transform from the camera to the base frame at the time included in the action goal's header, and then compute the food frame relative to that.
(Note, there is a separate question of whether it is desirable to have the app re-call the robot action if the page is refreshed, and, if not, how we can architect the app to avoid that. However, the above seems like a good improvement to make to acquisition code regardless, and we can also make the change to app architecture if needed.)
BTW @egordon friendly reminder about this. As you are changing to the new Mask type that now returns an RGB image, this might be an easy change to make.
Currently, the
AcquireFood
action assumes the camera is in the same pose as it was when the mask was captured from the web app. Although that is mostly true, the one case where it might not be is if the app user refreshes the page while the robot is moving. As the page is unmounting, the app will terminate the action, and then as it is re-mounting, it will call the action with the same parameters as when it was originally called. That is because the React app cannot differentiate between the page mounting when the user just transitions to it, versus the page mounting when it refreshes, and the "robot motion" pages are designed to call the robot action as soon as they mount.To address this, when AcquireFood computes the food frame, it should first use the TF tree to get the transform from the camera to the base frame at the time included in the action goal's header, and then compute the food frame relative to that.
(Note, there is a separate question of whether it is desirable to have the app re-call the robot action if the page is refreshed, and, if not, how we can architect the app to avoid that. However, the above seems like a good improvement to make to acquisition code regardless, and we can also make the change to app architecture if needed.)