iCubWaterloo01 S/N:044 – "MC: AEA encoder invalid data" error in shoulder joint

[paliasgh]

Robot Name 🤖

iCubWaterloo01 S/N:044

Request/Failure description

Hello,

This morning, our robot ran into a new problem. Almost every time, after the first movement of robot's left arm, the arm goes into HF mode with the error: "MC: AEA encoder invalid data. Hardware problem in the magnetic position sensor of the joint caused invalid position readings. (Joint=l_shoulder_yaw (NIB=2), encoderPort=0)". yarprobotinterface continues running with other parts.

I think it is a common issue (reported many times in the past), but looking at old issues I could not find the right solution for our robot, so I am posting a new issue. It might be another cable or connector issue.

Thanks in advance for your support.

Detailed context

Here is a yarprobotinterface log file:

yarprunlog_04_06_2024_15_01_01.log

(Some errors contain the label 'right' shoulder, but in fact, it looks like everything is with the 'left' shoulder. Right shoulder is doing okay!)

yarpmotorgui screenshot after the fault:

And before the fault:

Additional context

No response

How does it affect you?

No response

[paliasgh]

Hello @AntonioConsilvio , I just wanted follow up on this issue. Thank you very much!

[AntonioConsilvio]

Hi @paliasgh, sorry for the wait time. Unfortunately it's been a period with several urgencies, but we will now provide you with the proper support.

AEA encoder invalid data means that a joint (or more) is no longer receiving correct values from the joint sensor (AEA is the name of the sensor).

The first 3 joints of the iCub shoulders are matched, which means that if one of the 3 goes HW-fault, the other two also go HW-fault.

So you have to figure out which AEA is malfunctioning.

To do this, I recommend starting the robot. When the calibration is finished and the three joints are in HW fault, start the yarpmotorgui and manually move the three joints (pitch, roll, yaw). While you are manually moving the joints look at the encoder info from the yarpmotorgui and probably one of the 3 joints will always be holding the same value even though you are moving the joint.

That will be the sensor that is not working.

Once you find out which sensor is not working (which could be the yaw joint as you suggested, but I would like to be sure) , I will give you more info on how to fix it!

[paliasgh]

Hello @AntonioConsilvio , many thanks!

I did the test as you suggested. I tried to move all three DOFs at the same time when the joint was in HW fault and as seen below, all encoders kept updating:

https://github.com/robotology/icub-tech-support/assets/59978262/3a125266-7ad3-49bb-be8b-72e745b96ac1

Then, I restarted yarpmotorgui and let the code cause the HW. This time, I recorded the instance when HW happens. It looks like Joint 0 was frozen:

https://github.com/robotology/icub-tech-support/assets/59978262/db60f52f-948f-433f-9da3-a5ce6a2630e0

Next, I again restarted the robot and this time manually moved each joint. HW did not happen this time:

https://github.com/robotology/icub-tech-support/assets/59978262/ca81e2cb-9041-4d12-8e16-d779513ba42d

The funny thing is, after that robot never went into that HW fault again! I worked with the robot for the past 45 mins and everything was okay. Can the problem be a cable that was stuck in a bad position and after I moved the joints into some awkward positions the cable was released? I will keep you posted if the issue happens again, but for now, it seems like robot is doing okay.

[AntonioConsilvio]

Hi @paliasgh, if moving it by hand the only encoder frozen is the pitch, it is probably the cause of the HW faults.

Sometimes it works sometimes it doesn't, probably because the connector (or crimp) is about to disconnect, but it isn't completely, so sometimes it works sometimes it doesn't.

The left shoulder pitch sensor is connected to an electronic board called EMS 10.0.1.1 (EB1).

I don't think the problem is on the sensor, I think it's more likely that the cause is on the EMS side.

The problem is that the EMS is under the backpack, so unfortunately you'll have to remove it again 😕 to get to the connector.

Once you have reached the connector, I recommend testing it with tweezers for crimp tightness!

Please, send us feedback and if you have any questions feel free to ask!