New idea about the behavior of the robot during executing command "read" and "play"

[yl-wang996]

Since the trajectory generation is deeply dependent on the music sequence which we placed on the whiteboard. So if we placed the magnate in some weird way that the robot cannot generate a feasible trajectory, it means the trajectory planning failed for this music sequence, and then we need to manually change the sequence on the whiteboard until a feasible trajectory comes out.

Based on the situation above, this is the current design we have, as far as I know:

1. The human says "read", the robot takes a picture from the whiteboard and recognizes the music sequence from it.
2. The human says "play", the robot will first try to generate a feasible trajectory, and then only if the trajectory generation sucessed, it will execute this trajectory. If the trajectory planning fails, give some feedback to the human, then the human will know she/he should adjust the magnate.

So, Since the trajectory planning deeply depends on the music sequence on the whiteboard, we don't move the trajectory planning ahead, just together with the vision part. That means for the human saying "read", the robot will take pictures recognize the music sequence, and also try to generate the trajectory. With this design, it saves one step of human and also could be more comfortable for human-robot interaction.

How about this idea?

[Juphex]

Did you talk about this today? @yunlong-wang-cn Maybe we should postpone this idea to when there is a stable version that runs?

[yl-wang996]

not yet, I just have to fix the PR first.

[yl-wang996]

since Flo is concerned about the internal mechanism of the robot manipulation, the new pipeline could break something inside. therfore close it.