Actuator 5 Fault Error - Possible Bug

[saulcarvalho]

Description

I am using a Kinova Gen3 robot which I have developed a large code which is operational, based on waypoint trajectories. I have an algorithm to generate the waypoint coordinates depending on the resolution that I want. Right now I am only using cartesian coordinates, and the movement is only in a plane in the Y and Z axes with the X axis value being constant. Thing is, for a trajectory whose waypoints are separated from 5mm or higher, it works well, but when I try to reduce it to 1mm that is where I start getting problems.

The robot seems to be able to position itself fairly well at different coordinates positioned 1mm from each other, but after a few waypoint iterations (around 20 or 30 something), the robot enters a fault state saying the actuator 5 is in fault and that the motor current threshold was passed. When I do the same thing for 5mm, the full code can execute without a problem. Funny thing is, I am not even using actuator 5 during these specific movements ...

Version

KortexAPI : 2.6.0

Kortex Device :

Steps to reproduce

Executing the cartesian waypoint example from the kortex python API with a step in Y of 1mm between coordinates for 30-ish ierations (e.g., waypoint creation iterating on Y, 1mm at a time) will reproduce the same error.

Expected behavior

Ideally I would want to do the trajectories without raising the error for a resolution of 1mm between coordinates.

[martinleroux]

Hi @saulcarvalho ,

I will try to replicate your issue as soon as I can find some time. In the meantime, I want to highlight that waypoints are not really designed for densely-defined trajectories. Not only is the maximum number of waypoints in a list fairly low (1000), but dense trajectories also tend to have sharper changes in direction, which can slow the robot down a lot to ensure that the acceleration limits are always respected. For dense trajectories, we instead recommend either low-level position control, or servoing using Twist Commands.

That being said, none of these issues would explain why a joint fault would be triggered by such a trajectory, so I will get back to you once I have looked deeper into it.

Cheers

[martinleroux]

I have just tried the following trajectory using the send_cartesian_waypoint_trajectory.py example (tested with and without optimal blending):

waypointsDefinition = ( (0.6,   0.0,  0.4,  0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.001, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.002, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.003, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.003, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.004, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.005, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.006, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.007, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.008, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.009, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.010, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.011, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.012, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.013, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.014, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.015, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.016, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.017, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.018, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.019, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.020, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.021, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.022, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.023, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.024, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.025, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.026, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.027, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.028, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.029, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.030, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z),
                                    (0.6,  0.031, 0.4, 0.0, kTheta_x, kTheta_y, kTheta_z)
                                    )

I was not able to replicate your issue even after replaying the trajectory a handful of times, but I think I have a pretty good idea as to what you are experiencing. As I mentioned above, Waypoints are really not designed for dense trajectories. At this level of density, the behaviour is even worse than I expected - to avoid overshooting points, the robot brakes at every waypoint and quickly starts moving again. In my setup, this induces a large amount of vibration in the robot. Depending on the orientation and the payload, this could easily trigger any joint fault between maximum velocity, maximum torque, maximum current or following error - current just happens to be the one that triggers first.

[saulcarvalho]

I get what you are saying. Let me give a bit more of context.

At the moment I am specifically using waypoint trajectories because I need to take measurements at each position. The payload is approximately 0.5 kg and I corrected its center of mass together with a Robotiq Gripper F85. At each position I am purposely stopping the robot for some seconds to perform the measurements and then go to each point (because the vibrations may affect the measurements).

In relation to what you said about overshooting, it does make a lot of sense, and that is exactly why I also limited the actuator speed from the default 0.5 m/s to around 10 cm/s, even so I didn't see much improvements.

I think I will give a try to change the part of the code to twist commands and see if it improves, and then try the low level commands. Do you have any other suggestion?

[martinleroux]

Within 1mm of travel, the robot still won't near 10cm/s of speed, hence why you aren't seeing any improvement.

If you are stopping the robot completely at every point, you may consider using Sequences or Reach Pose actions (see examples 102/01 and 102/02). These use a somewhat less aggressive acceleration value and may help with the issue. Try with velocity constraints around 0.01-0.05 m/s.

[saulcarvalho]

Reach Pose actions is exactly what I am using.

It seems that controling both joint speeds (around 20 to 30 degree/s) and linear speed (around 5 mm/s) to lower values improves the operation of the robot for some time (using control config), but after a few waypoints the same error happens. I don't think sequence actions are going to work well for the measurements that I'm doing, unfortunately.

[martinleroux]

Oh, I assumed you used waypoints because you said so in your problem description and then closed the ticket since you only replied with a thumbs up to my last message.

If you are already using ReachPose, then Sequences won't improve the behaviour anyway. You could try using actual waypoints. The maximum acceleration is a bit higher but the acceleration profile is smoother (trapezoidal instead of bang-bang). If this still does not resolve your issue, the other alternatives I can think of are much more complicated, such as implementing a joint impedance controller to let J5 wiggle around its position when vibrated instead of locking it in place. Beyond that, I am afraid there isn't much I can do.

I will reopen the issue and leave it open until I hear back from you.