servoOn not turning servos on

The same user who reported the issue https://github.com/tork-a/rtmros_nextage/issues/332 is experiencing that the servoOn command does not turn the servos on, claiming "Once the servos are off for whatever reason, I cannot turn them back on. The only thing that helps is to use the power switch on the QNX and restart."

move the arms to the safe pose (i.e. no obstructing along the paths back to the work area), then you can run servoOn to fire up all servos. Make sure you turn off the emergency switch that prioritizes over any Python commands.

This did not help.

Logs and comments on Python i/f:

[hrpsys.py] servo on failed.
[hrpsys.py] exception occured
Move to Actual State, Just a minute.
Turn on Hand Servo
Hands Servo on: True
However, robot.isServoOn() is still False and also the arms have not been moved at all. The red light on the robot is blinking, and depending on the hand switch position, the orange light as well.

Logs from QNX

$ rosversion hrpsys
315.12.1
$ rosversion hironx_ros_bridge
1.1.23

ModelLoader.log. I see something unusual at the end but I'm not sure if it's related.

ready
loading /opt/jsk/etc/HIRONX/model/main.wrl
Humanoid node
Joint nodeWAIST
  Segment node WAIST_Link
  Joint nodeCHEST_JOINT0
    Segment node CHEST_JOINT0_Link
    Joint nodeHEAD_JOINT0
      Segment node HEAD_JOINT0_Link
      Joint nodeHEAD_JOINT1
        Segment node HEAD_JOINT1_Link
        VisionSensorCAMERA_HEAD_R
        VisionSensorCAMERA_HEAD_L
    Joint nodeRARM_JOINT0
      Segment node RARM_JOINT0_Link
      Joint nodeRARM_JOINT1
        Segment node RARM_JOINT1_Link
        Joint nodeRARM_JOINT2
          Segment node RARM_JOINT2_Link
          Joint nodeRARM_JOINT3
            Segment node RARM_JOINT3_Link
            Joint nodeRARM_JOINT4
              Segment node RARM_JOINT4_Link
              Joint nodeRARM_JOINT5
                Segment node RARM_JOINT5_Link
    Joint nodeLARM_JOINT0
      Segment node LARM_JOINT0_Link
      Joint nodeLARM_JOINT1
        Segment node LARM_JOINT1_Link
        Joint nodeLARM_JOINT2
          Segment node LARM_JOINT2_Link
          Joint nodeLARM_JOINT3
            Segment node LARM_JOINT3_Link
            Joint nodeLARM_JOINT4
              Segment node LARM_JOINT4_Link
              Joint nodeLARM_JOINT5
                Segment node LARM_JOINT5_Link
Node is inconvertible and removed from the scene graph
Node is inconvertible and removed from the scene graph
Warning: Mass is zero. <Model>HiroNX <Link>CHEST_JOINT0
Warning: Mass is zero. <Model>HiroNX <Link>WAIST
The model was successfully loaded ! 
cache found for /opt/jsk/etc/HIRONX/model/main.wrl
cache found for /opt/jsk/etc/HIRONX/model/main.wrl
cache found for /opt/jsk/etc/HIRONX/model/main.wrl
cache found for /opt/jsk/etc/HIRONX/model/main.wrl
cache found for /opt/jsk/etc/HIRONX/model/main.wrl
cache found for /opt/jsk/etc/HIRONX/model/main.wrl
loading 
 cannot be found.
Retrying to load the file as a standard VRML file
 cannot be found.
loading failed.
 cannot be found.

rtcd.log

:
Upper joint limit error LARM_JOINT0
Upper joint limit error LARM_JOINT2
Upper joint limit error LARM_JOINT3
Upper joint limit error LARM_JOINT4
Upper joint limit error LARM_JOINT4
Upper joint limit error LARM_JOINT4
Upper joint limit error LARM_JOINT4
Upper joint limit error LARM_JOINT4
Upper joint limit error LARM_JOINT4
Upper joint limit error LARM_JOINT4
Upper joint limit error LARM_JOINT4
error limit over RARM_JOINT2(5), qRef=0, qCurrent=-2.26024 , Error=2.26024 > 0.18 (limit), servo_state = ON, q=-2.08024
error limit over RARM_JOINT3(6), qRef=0, qCurrent=0.263023 , Error=-0.263023 > 0.18 (limit), servo_state = ON, q=0.083023
error limit over LARM_JOINT2(11), qRef=0, qCurrent=-2.26253 , Error=2.26253 > 0.18 (limit), servo_state = ON, q=-2.08253
error limit over LARM_JOINT3(12), qRef=0, qCurrent=-0.264481 , Error=0.264481 > 0.18 (limit), servo_state = ON, q=-0.0844814
Timeover: processing time =  6.0[ms]
 0.0,  1.0,  0.0,  0.0,  0.0,  5.0,  0.0,  0.0, 
error limit over RARM_JOINT2(5), qRef=0, qCurrent=-2.26024 , Error=2.26024 > 0.18 (limit), servo_state = ON, q=-2.08024
error limit over RARM_JOINT3(6), qRef=0, qCurrent=0.263023 , Error=-0.263023 > 0.18 (limit), servo_state = ON, q=0.083023
error limit over LARM_JOINT2(11), qRef=-0.0162838, qCurrent=-2.26253 , Error=2.24625 > 0.18 (limit), servo_state = ON, q=-2.08253
error limit over LARM_JOINT3(12), qRef=0.0317515, qCurrent=-0.264481 , Error=0.296233 > 0.18 (limit), servo_state = ON, q=-0.0844814
error limit over RARM_JOINT2(5), qRef=0, qCurrent=-2.26024 , Error=2.26024 > 0.18 (limit), servo_state = ON, q=-2.08024
error limit over RARM_JOINT3(6), qRef=0, qCurrent=0.263023 , Error=-0.263023 > 0.18 (limit), servo_state = ON, q=0.083023
error limit over LARM_JOINT2(11), qRef=-0.107084, qCurrent=-2.26253 , Error=2.15545 > 0.18 (limit), servo_state = ON, q=-2.08253
error limit over LARM_JOINT3(12), qRef=0.208801, qCurrent=-0.264481 , Error=0.473283 > 0.18 (limit), servo_state = ON, q=-0.0844814
error limit over RARM_JOINT2(5), qRef=0, qCurrent=-2.25978 , Error=2.25978 > 0.18 (limit), servo_state = ON, q=-2.07978
error limit over RARM_JOINT3(6), qRef=0, qCurrent=0.262424 , Error=-0.262424 > 0.18 (limit), servo_state = ON, q=0.0824237
error limit over LARM_JOINT2(11), qRef=-0.291508, qCurrent=-2.26207 , Error=1.97056 > 0.18 (limit), servo_state = ON, q=-2.08207
error limit over LARM_JOINT3(12), qRef=0.568405, qCurrent=-0.263902 , Error=0.832307 > 0.18 (limit), servo_state = ON, q=-0.083902
error limit over LARM_JOINT4(13), qRef=0.193237, qCurrent=-0.00312932 , Error=0.196367 > 0.18 (limit), servo_state = ON, q=0.176871
Timeover: processing time = 10.0[ms]
 0.0,  0.0,  0.0,  0.0,  0.0, 10.0,  0.0,  0.0, 
error limit over LARM_JOINT5(14), qRef=-0.299952, qCurrent=-0.00125572 , Error=-0.298696 > 0.18 (limit), servo_state = ON, q=-0.181256

(Entire rtcd.log BEFORE and AFTER the issue happened.

start-jsk / rtmros_hironx

servoOn not turning servos on #507