cube orange losing RC on v1.12 beta: [px4io] Failed to communicate with IO

[ArkadiuszNiemiec]

Describe the bug We have updated the PX4 to 4th beta of v1.12 from master branch of v1.11. After few reboots and flights the PX4 looses RC data: there is no longer a RC mavlink message, QGC doesn't show any data in Radio tab etc. Console output of px4io status shows that it is not running or shows data but prints it very slowly (one line per second?).

nsh> px4io status
ERROR [px4io] not started
nsh> px4io start
ERROR [px4io] Failed to communicate with IO, abort.
ERROR [px4io] driver init failed
nsh> 

nsh> px4io status
WARN  [px4io] loaded
protocol 2147483648 hardware 2147483648 bootloader 2147483648 buffer 2147483648B crc 0x8000000080000000
2147483648 controls 2147483648 actuators 2147483648 R/C inputs 2147483648 analog inputs 2147483648 relays
 px4io_status_s
    timestamp: 745597  (403.156963 seconds ago)
    voltage_v: 0.0000
    rssi_v: 1.5210
    free_memory_bytes: 1224
    actuators: [-10000, -10000, -10000, -10000, -10000, -10000, -10000, -10000]
    servos: [900, 900, 900, 900, 900, 900, 900, 900]
    raw_inputs: [1502, 1504, 1382, 1494, 2015, 1504, 1504, 991, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    status_outputs_armed: False
    status_override: False
    status_rc_ok: True
    status_rc_ppm: True
    status_rc_dsm: False
    status_rc_sbus: False
    status_fmu_ok: False
    status_raw_pwm: False
    status_mixer_ok: True
    status_arm_sync: True
    status_init_ok: True
    status_failsafe: False
    status_safety_off: True
    status_fmu_initialized: True
    status_rc_st24: False
    status_rc_sumd: False
    alarm_vbatt_low: False
    alarm_temperature: False
    alarm_servo_current: False
    alarm_acc_current: False
    alarm_fmu_lost: True
    alarm_rc_lost: True
    alarm_pwm_error: False
    alarm_vservo_fault: False
    arming_io_arm_ok: True
    arming_fmu_armed: False
    arming_fmu_prearmed: False
    arming_manual_override_ok: False
    arming_failsafe_custom: True
    arming_inair_restart_ok: True
    arming_always_pwm_enable: True
    arming_rc_handling_disabled: False
    arming_lockdown: False
    arming_force_failsafe: False
    arming_termination_failsafe: False
    arming_override_immediate: True

reversed outputs: [________] trims: r:   0.0000 p:   0.0000 y:   0.0000
0 raw R/C inputs
R/C flags: 0x0000
mapped R/C inputs 0x0000
ADC inputs
features 0x0000
rates 0x80000000 default 2147483648 alt 2147483648 sbus 2147483648
debuglevel 2147483648
controls 0: 0 0 0 0 0 0 0 0
controls 1: 0 0 0 0 0 0 0 0
controls 2: 0 0 0 0 0 0 0 0
controls 3: 0 0 0 0 0 0 0 0
input 0 min 2147483648 center 2147483648 max 2147483648 deadzone 2147483648 assigned 2147483648 options 0x0000
input 1 min 2147483648 center 2147483648 max 2147483648 deadzone 2147483648 assigned 2147483648 options 0x0000
input 2 min 2147483648 center 2147483648 max 2147483648 deadzone 2147483648 assigned 2147483648 options 0x0000
input 3 min 2147483648 center 2147483648 max 2147483648 deadzone 2147483648 assigned 2147483648 options 0x0000
input 4 min 2147483648 center 2147483648 max 2147483648 deadzone 2147483648 assigned 2147483648 options 0x0000
input 5 min 2147483648 center 2147483648 max 2147483648 deadzone 2147483648 assigned 2147483648 options 0x0000
input 6 min 2147483648 center 2147483648 max 2147483648 deadzone 2147483648 assigned 2147483648 options 0x0000
input 7 min 2147483648 center 2147483648 max 2147483648 deadzone 2147483648 assigned 2147483648 options 0x0000
input 8 min 2147483648 center 2147483648 max 2147483648 deadzone 2147483648 assigned 2147483648 options 0x0000
input 9 min 2147483648 center 2147483648 max 2147483648 deadzone 2147483648 assigned 2147483648 options 0x0000
input 10 min 2147483648 center 2147483648 max 2147483648 deadzone 2147483648 assigned 2147483648 options 0x0000
input 11 min 2147483648 center 2147483648 max 2147483648 deadzone 2147483648 assigned 2147483648 options 0x0000
input 12 min 2147483648 center 2147483648 max 2147483648 deadzone 2147483648 assigned 2147483648 options 0x0000
input 14 min 2147483648 center 2147483648 max 2147483648 deadzone 2147483648 assigned 2147483648 options 0x0000
input 15 min 2147483648 center 2147483648 max 2147483648 deadzone 2147483648 assigned 2147483648 options 0x0000
input 16 min 2147483648 center 2147483648 max 2147483648 deadzone 2147483648 assigned 2147483648 options 0x0000
input 17 min 2147483648 center 2147483648 max 2147483648 deadzone 2147483648 assigned 2147483648 options 0x0000
failsafe 2147483648 2147483648 2147483648 2147483648 2147483648 2147483648 2147483648 2147483648
disarmed values 2147483648 2147483648 2147483648 2147483648 2147483648 2147483648 2147483648 2147483648
IMU heater off

Power cycling the cube solves the problem.

To Reproduce Steps to reproduce the behavior:

1. Fly
2. Reboot without power cycly
3. Try to change mode or look into rc data

Expected behavior A PX4 to maintain RC connection even after hundreds of reset or restart :)

Drone (please complete the following information):

• MC
• PX4 v1.12.0 beta 4

[ArkadiuszNiemiec]

It looks like it's connected with #17093. @LorenzMeier can you take a look?

[LorenzMeier]

Looks to me like your IO would need to updated and didn't successfully automatically. @dagar could you look into this?

[ArkadiuszNiemiec]

It just happened again on v12 beta 4:

nsh> px4io stop
nsh> px4io start
ERROR [px4io] Failed to communicate with IO, abort.
ERROR [px4io] driver init failed