Console hangs/ErrorHandler is invoked during usb console usage.

[adam-at-epsilon]

We have some issues when using the USB console, for instance when using the sensor sampling:

1) The ErrorHandler is invoked (all LEDs are alight) 2) The text stops flowing to the screen.

[adam-at-epsilon]

When this happens, #define FCB_ACCMAG_DEBUG is set and GPIO pin PD11 is high, which means that we are in the process of fetching data from the magnetometer. (one try)

Behaviour seen with PD9 high, indicating that data is being fetched from accelerometer.

Slowing down output data rate of

• magnetometer from 75 to 30 Hz doesn't help.
• and accelerometer from 50 to 25Hz doesn't help.

[stenbergd]

Nice work debugging! Not sure why this is occuring though since it often works well for prolonged periods of time. Perhaps the sensors have strange behaviour due to power supply issues from USB?

I've been thinking about this sensor queue - is it possible that multiple unserviced queue items from the same sensor can enter the queue? If so, maybe we could "peek" into the queue and replace that item? This way we'll ignore old sensor values in the queue if a newer one has been read.

[adam-at-epsilon]

@DanielStenis the main suspect at the moment is the I2C bus "hanging" for some reason and the I2C timeout is long by default. Since the I2C timeout is longer than the poll intervals of the sensors, a single I2C error blocks the thread for all sensors.

Power supply could perhaps be an issue ... but a check with task-status reveals that during task sampling the IDLE tasks shows 88%.

By shortening the timeout, I saw the I2C HAL SW reset itself.

It is strange I2C acts up, we're effectively only using the I2C sequentially since it is only called from a single thread.

[stenbergd]

I see. Maybe some sort of power glitch is enough to make the I2C bus act weirdly. It seems to happen when we move the board around, where you'd imagine the USB plug to twitch around in the connector.

But maybe that sounds a bit far-fetched. We could try it with a more stable power source, plugging in a 5V source to the 5V pin. I have an adjustable power supply unit on my desk, feel free to use it if you think it is relevant.

Also, when we eventually get around to https://github.com/afconsult-south/dragonfly-fcb/issues/7 improvement of sensor reading/handling is to be expected.

[adam-at-epsilon]

General FreeRTOS behaviour: queues and semaphores unblock tasks in priority order, not FIFO order as is recommended in for instance VxWorks.

[adam-at-epsilon]

Just leaving the card on my desk overnight, with 2 tick timeout and the debugger breakpoint set in I2Cx_Error was never hit. This indicates the I2C bus is stable when the USB console is not used.

[adam-at-epsilon]

Ran another overnight test, this time with continuous USB printouts. I put an error counter on the I2Cx_Reset function in the "systime" and it returned zero, so the card has run the I2C bus for 15 hours without I2C errors. So the I2C communication looks stable.

The next candidate to eliminate is power supply problems due to imprecise USB contact connection.

[adam-at-epsilon]

New hypothesis: I think I have found one root cause for the I2C hanging ... when touching the PB7 pin (I2C1 SDA) it hangs instantly. I2C1 is the bus we use for LSM303DLHC communication.

This easily happens when we move the board about by hand and also explains why it's so stable when the board is left on the desk.

Care should be taken when we are assembling the board in the quadcopter so the pins cannot be accidentally touched by hand.

[stenbergd]

Wow, nice work debugging this!

[adam-at-epsilon]

Thanks :+1: I discovered it while using the oscilloscope probe on PB7 while working on something else and then the same error was trigged by using fingers. The probe on PB6 (I2C1 SCL) but not PB7. So I think it has become a hardware issue rather than a code stability issue.

Is there an issue list for hardware?

[adam-at-epsilon]

Closing this. Re-open if it comes back.