ihmcrobotics / ihmc_valkyrie_ros

IHMC ROS integration for Valkyrie robot
Apache License 2.0
7 stars 10 forks source link

IndexOutOfBoundsException exception #14

Closed v-lopez closed 6 years ago

v-lopez commented 7 years ago

Using latest srcsim installation

Steps to reproduce issue

Have only seen it once after heavy use of the robot, while recording rosbags for issue https://github.com/ihmcrobotics/ihmc_valkyrie_ros/issues/13 I finished recording a rosbag sending trajectories only to Left arm using OVERRIDE mode, and restarted my application publishing Left & Right trajectories using QUEUE mode.

Probably a spurious failure, nevertheless here's the stacktrace in case it's easy to spot.

[ INFO] [1493794787.603146262, 231.455000000]: [WARN] (ControllerCommandValidationTools.java:67): Joint out of bounds: upperNeckPitch (-0.872, 0.0) = 1.9255966560190356E-22 (t=0)
[ INFO] [1493794787.603193078, 231.455000000]: [WARN] (ControllerCommandValidationTools.java:50): Invalid joint trajectory ( 2 - upperNeckPitch)
[ERROR] [1493794787.603205447, 231.455000000]: Neck command out of joint limits!
[ INFO] [1493794787.667847438, 231.471000000]: [WARN] (ControllerCommandValidationTools.java:67): Joint out of bounds: upperNeckPitch (-0.872, 0.0) = 5.788150776782637E-22 (t=0)
[ INFO] [1493794787.667922390, 231.471000000]: [WARN] (ControllerCommandValidationTools.java:50): Invalid joint trajectory ( 2 - upperNeckPitch)
[ERROR] [1493794787.667941960, 231.471000000]: Neck command out of joint limits!
[ INFO] [1493794787.741492413, 231.487000000]: [WARN] (ControllerCommandValidationTools.java:67): Joint out of bounds: upperNeckPitch (-0.872, 0.0) = 1.7839040744451672E-21 (t=0)
[ INFO] [1493794787.741548979, 231.487000000]: [WARN] (ControllerCommandValidationTools.java:50): Invalid joint trajectory ( 2 - upperNeckPitch)
[ERROR] [1493794787.741568715, 231.487000000]: Neck command out of joint limits!
[ INFO] [1493794787.874641883, 231.515000000]: [WARN] (ControllerCommandValidationTools.java:67): Joint out of bounds: upperNeckPitch (-0.872, 0.0) = 8.899306953800144E-22 (t=0)
[ INFO] [1493794787.874714346, 231.515000000]: [WARN] (ControllerCommandValidationTools.java:50): Invalid joint trajectory ( 2 - upperNeckPitch)
[ERROR] [1493794787.874746485, 231.515000000]: Neck command out of joint limits!
[ERROR] [1493794787.877122253, 231.515000000]: Exception in thread "SynchronousMultiThreadedRobotController-thread-1" java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
[ERROR] [1493794787.877189337, 231.515000000]:  at us.ihmc.wholeBodyController.DRCControllerThread.run(DRCControllerThread.java:360)
[ERROR] [1493794787.877205628, 231.515000000]:  at us.ihmc.wholeBodyController.concurrent.SynchronousMultiThreadedRobotController$ReentrantLockedControlElementRunner.run(SynchronousMultiThreadedRobotController.java:89)
[ERROR] [1493794787.877216062, 231.515000000]:  at java.lang.Thread.run(Thread.java:745)
[ERROR] [1493794787.877319762, 231.515000000]: Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
[ERROR] [1493794787.877335405, 231.515000000]:  at us.ihmc.robotics.lists.RecyclingArrayList.rangeCheck(RecyclingArrayList.java:340)
[ERROR] [1493794787.877345215, 231.515000000]:  at us.ihmc.robotics.lists.RecyclingArrayList.get(RecyclingArrayList.java:130)
[ERROR] [1493794787.877356132, 231.515000000]:  at us.ihmc.robotics.math.trajectories.waypoints.SimpleTrajectoryPointList.getTrajectoryPoint(SimpleTrajectoryPointList.java:59)
[ERROR] [1493794787.877368137, 231.515000000]:  at us.ihmc.commonWalkingControlModules.trajectories.LookAheadCoMHeightTrajectoryGenerator.initializeOffsetTrajectoryGenerator(LookAheadCoMHeightTrajectoryGenerator.java:893)
[ERROR] [1493794787.877378542, 231.515000000]:  at us.ihmc.commonWalkingControlModules.trajectories.LookAheadCoMHeightTrajectoryGenerator.solve(LookAheadCoMHeightTrajectoryGenerator.java:762)
[ERROR] [1493794787.877392839, 231.515000000]:  at us.ihmc.commonWalkingControlModules.trajectories.LookAheadCoMHeightTrajectoryGenerator.solve(LookAheadCoMHeightTrajectoryGenerator.java:728)
[ERROR] [1493794787.877406820, 231.515000000]:  at us.ihmc.commonWalkingControlModules.instantaneousCapturePoint.CenterOfMassHeightManager.solve(CenterOfMassHeightManager.java:177)
[ERROR] [1493794787.877417233, 231.515000000]:  at us.ihmc.commonWalkingControlModules.instantaneousCapturePoint.CenterOfMassHeightManager.computeDesiredCoMHeightAcceleration(CenterOfMassHeightManager.java:198)
[ERROR] [1493794787.877431249, 231.515000000]:  at us.ihmc.commonWalkingControlModules.highLevelHumanoidControl.highLevelStates.WalkingHighLevelHumanoidController.updateManagers(WalkingHighLevelHumanoidController.java:586)
[ERROR] [1493794787.877445857, 231.515000000]:  at us.ihmc.commonWalkingControlModules.highLevelHumanoidControl.highLevelStates.WalkingHighLevelHumanoidController.doAction(WalkingHighLevelHumanoidController.java:518)
[ERROR] [1493794787.877456943, 231.515000000]:  at us.ihmc.robotics.stateMachines.conditionBasedStateMachine.GenericStateMachine.doAction(GenericStateMachine.java:152)
[ERROR] [1493794787.877467322, 231.515000000]:  at us.ihmc.commonWalkingControlModules.highLevelHumanoidControl.HighLevelHumanoidControllerManager.doControl(HighLevelHumanoidControllerManager.java:178)
[ERROR] [1493794787.877477776, 231.515000000]:  at us.ihmc.robotics.robotController.ModularRobotController.doControl(ModularRobotController.java:20)
[ERROR] [1493794787.877487699, 231.515000000]:  at us.ihmc.wholeBodyController.DRCControllerThread.run(DRCControllerThread.java:352)
[ERROR] [1493794787.877499825, 231.515000000]:  ... 2 more

If it ever happens again I'll expand this.

v-lopez commented 7 years ago

Happened 2 more times, I launched the simulation with the ihmc controllers and send only sent the commands attached in the rosbag.

To me it looks related to QUEUE mode, because I had never experienced this before, and it's happening a lot since I'm trying to use QUEUE mode.

index_out_of_bounds.zip

INIT: Switch to high level control
publishing and latching message for 3.0 seconds
INIT: Wait 2s (ROS time) for control to be switched
INIT: Detach from harness
INIT: Done
[startup_robot-13] process has finished cleanly
log file: /home/user/.ros/log/23bff06a-2fd1-11e7-bc8d-0242ac110003/startup_robot-13*.log
publishing and latching message for 3.0 seconds
[ INFO] [1493796194.282622033, 41.056000000]: [WARN] (ControllerCommandValidationTools.java:67): Joint out of bounds: upperNeckPitch (-0.872, 0.0) = 1.9265062059460516E-12 (t=0)
[ INFO] [1493796194.283008525, 41.056000000]: [WARN] (ControllerCommandValidationTools.java:50): Invalid joint trajectory ( 2 - upperNeckPitch)
[ERROR] [1493796194.283220733, 41.056000000]: Neck command out of joint limits!
[ INFO] [1493796194.455819847, 41.099000000]: [WARN] (ControllerCommandValidationTools.java:67): Joint out of bounds: upperNeckPitch (-0.872, 0.0) = 3.013319532142589E-21 (t=0)
[ INFO] [1493796194.455881973, 41.099000000]: [WARN] (ControllerCommandValidationTools.java:50): Invalid joint trajectory ( 2 - upperNeckPitch)
[ERROR] [1493796194.455909172, 41.099000000]: Neck command out of joint limits!
[ INFO] [1493796194.724013997, 41.184000000]: [WARN] (ControllerCommandValidationTools.java:67): Joint out of bounds: upperNeckPitch (-0.872, 0.0) = 1.5119536331343411E-21 (t=0)
[ INFO] [1493796194.724118986, 41.184000000]: [WARN] (ControllerCommandValidationTools.java:50): Invalid joint trajectory ( 2 - upperNeckPitch)
[ERROR] [1493796194.724151450, 41.184000000]: Neck command out of joint limits!
[ INFO] [1493796194.773120920, 41.200000000]: [WARN] (ControllerCommandValidationTools.java:67): Joint out of bounds: upperNeckPitch (-0.872, 0.0) = 2.0858184548991296E-21 (t=0)
[ INFO] [1493796194.773230759, 41.200000000]: [WARN] (ControllerCommandValidationTools.java:50): Invalid joint trajectory ( 2 - upperNeckPitch)
[ERROR] [1493796194.773263200, 41.200000000]: Neck command out of joint limits!
[ERROR] [1493796194.774649133, 41.200000000]: Exception in thread "SynchronousMultiThreadedRobotController-thread-1" java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
[ERROR] [1493796194.774868219, 41.200000000]:   at us.ihmc.wholeBodyController.DRCControllerThread.run(DRCControllerThread.java:360)
[ERROR] [1493796194.775012225, 41.200000000]:   at us.ihmc.wholeBodyController.concurrent.SynchronousMultiThreadedRobotController$ReentrantLockedControlElementRunner.run(SynchronousMultiThreadedRobotController.java:89)
[ERROR] [1493796194.775141955, 41.200000000]:   at java.lang.Thread.run(Thread.java:745)
[ERROR] [1493796194.775410554, 41.200000000]: Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
[ERROR] [1493796194.775504051, 41.200000000]:   at us.ihmc.robotics.lists.RecyclingArrayList.rangeCheck(RecyclingArrayList.java:340)
[ERROR] [1493796194.775635359, 41.200000000]:   at us.ihmc.robotics.lists.RecyclingArrayList.get(RecyclingArrayList.java:130)
[ERROR] [1493796194.775775971, 41.200000000]:   at us.ihmc.robotics.math.trajectories.waypoints.SimpleTrajectoryPointList.getTrajectoryPoint(SimpleTrajectoryPointList.java:59)
[ERROR] [1493796194.775916582, 41.200000000]:   at us.ihmc.commonWalkingControlModules.trajectories.LookAheadCoMHeightTrajectoryGenerator.initializeOffsetTrajectoryGenerator(LookAheadCoMHeightTrajectoryGenerator.java:893)
[ERROR] [1493796194.776055573, 41.200000000]:   at us.ihmc.commonWalkingControlModules.trajectories.LookAheadCoMHeightTrajectoryGenerator.solve(LookAheadCoMHeightTrajectoryGenerator.java:762)
[ERROR] [1493796194.776165470, 41.200000000]:   at us.ihmc.commonWalkingControlModules.trajectories.LookAheadCoMHeightTrajectoryGenerator.solve(LookAheadCoMHeightTrajectoryGenerator.java:728)
[ERROR] [1493796194.776228326, 41.200000000]:   at us.ihmc.commonWalkingControlModules.instantaneousCapturePoint.CenterOfMassHeightManager.solve(CenterOfMassHeightManager.java:177)
[ERROR] [1493796194.776287671, 41.200000000]:   at us.ihmc.commonWalkingControlModules.instantaneousCapturePoint.CenterOfMassHeightManager.computeDesiredCoMHeightAcceleration(CenterOfMassHeightManager.java:198)
[ERROR] [1493796194.776635251, 41.200000000]:   at us.ihmc.commonWalkingControlModules.highLevelHumanoidControl.highLevelStates.WalkingHighLevelHumanoidController.updateManagers(WalkingHighLevelHumanoidController.java:586)
[ERROR] [1493796194.776702157, 41.200000000]:   at us.ihmc.commonWalkingControlModules.highLevelHumanoidControl.highLevelStates.WalkingHighLevelHumanoidController.doAction(WalkingHighLevelHumanoidController.java:518)
[ERROR] [1493796194.776781446, 41.200000000]:   at us.ihmc.robotics.stateMachines.conditionBasedStateMachine.GenericStateMachine.doAction(GenericStateMachine.java:152)
[ERROR] [1493796194.776963058, 41.200000000]:   at us.ihmc.commonWalkingControlModules.highLevelHumanoidControl.HighLevelHumanoidControllerManager.doControl(HighLevelHumanoidControllerManager.java:178)
[ERROR] [1493796194.777059071, 41.200000000]:   at us.ihmc.robotics.robotController.ModularRobotController.doControl(ModularRobotController.java:20)
[ERROR] [1493796194.777094082, 41.200000000]:   at us.ihmc.wholeBodyController.DRCControllerThread.run(DRCControllerThread.java:352)
[ERROR] [1493796194.777214268, 41.200000000]:   ... 2 more
dljsjr commented 7 years ago

@v-lopez We'll dig in to this a bit but my guess is that this is related to some sort of buffer wraparound that we don't normally see because we don't normally send commands to our API at 100Hz.