Richard-Gemmell / teensy4_i2c

An I2C library for the Teensy 4. Provides slave and master mode.
MIT License
92 stars 19 forks source link

I2C slave hangs if digitalWrite() is used #30

Closed djholt closed 1 year ago

djholt commented 1 year ago

Hi Richard,

Thanks for your awesome work on this project. I'm trying to use a Teensy 4.1 as an I2C slave device connected to a proprietary master device, where the slave only reads data sent from the master. I've had good success with an Arduino Due, but I'm having some hanging issues with a Teensy 4.1. I'm hoping you can help point me in the right direction. Here's what I'm observing.

Occasionally, the I2C communication will stall. When stalled, SDA is kept low by the Teensy and no further bytes are read. I've finally determined that this will happen when I digitalWrite() to a completely unrelated GPIO pin (any pin, it seems) while I2C data is actively being received/read. This all happens in my loop(), and I've taken care to avoid any delay()-like operations, just in case that's an issue. My receiveEvent() function is included below. If I omit the digitalWrite() calls, the hanging stops entirely. Of note, this happens using the native Wire.h as well as i2c_driver_wire.h. The presence of external pullup resistors doesn't seem to make any difference. The stuck_bus and dev branches also produce the issue.

void receiveEvent(int howMany) { c = Wire1.read(); }

Any input would be much appreciated!

Richard-Gemmell commented 1 year ago

Hi djholt,

This stall is generally known as a "stuck bus". It happens when the master and slave get out of sync. In your case the slave is trying to send a 0 and is waiting for the master to pulse the clock (SCL). Unfortunately, the master isn't expecting a bit for some reason. The master is probably trying to start a new I2C message but it thinks someone else is using the bus because SDA is low. Deadlock!

As a work around, you can "reset" the stuck bus on the Teensy by calling stop_listening() on the slave and then listen() again. You could probably set up a watchdog timer to track when you last received a message and do the reset if it times out. A more complex solution is to monitor the I2C bus directly. That's probably overkill in your case though.

There are lots of possible causes. To rule them out:

It's possible that the master implementation does not support "clock stretching". If it's bit banging rather than using hardware support then it may well be a bit sensitive especially if it's handling other IO during an I2C message.

I've created the issue-30 with a couple of tweaks to the driver that might help. See this diff. Note that you MUST modify this file unless the bus speed is set to 100 kHz. It won't work as is at higher bus speeds.

The release version of the driver allows the slave to stretch the clock. I've disabled this by commenting out line 449. That might help if the master doesn't support clock stretching. This could well be the problem given that you are seeing the lockup when you write to GPIO.

I've added line 452 to make the slave wait a bit before changing the value of SDA. At the moment the slave is free to change SDA a few nanoseconds after the master drops SCL. This might confuse the master. If you're running at 400k or 1MHz you'll need to comment out line 452 and pick one of the lines underneath.

I've added line 457 to enable the slave glitch filter. This should make the slave less sensitive to noise. Like the change above, you'll need to pick the right line to suit your bus speed.

FWIW the last 2 changes may well be included in the release driver in the future.

I'd be curious to know:

I'd be most grateful if you could try to reproduce this fault with just the Teensy talking to itself. If you can, then please send me the source code. That'll enable me to find out exactly what's going on.

cheers, Richard

djholt commented 1 year ago

Thank you so much for taking the time to provide a detailed answer and a testing branch. I'll take some time to experiment with each of these changes over the next few days. In the meantime, here are a few initial answers to your questions:

what bus speed you're running

The clock frequency output by the master is consistently 250kHz (not configurable). Based on this, how should I be configuring the clock speed on the slave?

if the GPIO that triggers the fault is connected to the master

Yes, it is connected to the master, but this same configuration works well when using an Arduino Due.

whether the master supports clock stretching

Unknown, but this is something I've been trying to determine. I have had zero success using an ESP32 as a slave, and I understand the original ESP32 does not support slave clock stretching, which may be one possible explanation. I'll try disabling clock stretching on the Teensy and see what happens.

whether the master is bit banging or using I2C hardware support

I strongly suspect the master device has a hardware I2C implementation based on its dated hardware and microcontroller.

Thanks again! I'll report back.

Richard-Gemmell commented 1 year ago

what bus speed you're running

the clock frequency output by the master is consistently 250kHz (not configurable). Based on this, how should I be configuring the clock speed on the slave?

Use the settings for 400 kHz.

if the GPIO that triggers the fault is connected to the master

yes it is connected to the master, but this same configuration works well when using an Arduino Due.

Hmmm. It's possible that there's a bug in the master code that's sensitive to the exact timing of interrupts or the I2C timings used. i.e. a race condition of some kind. The Due and the Teensy might be different in some significant way. You could run the system so that the Teensy is still setting the GPIO but it's connected to something other than the master. If it still locks up then the issue is only in the Teensy code.

If you can figure out a way to avoid setting the GPIO during a I2C message then that'll probably fix it. If the master is requesting data from the slave then the I2CSlave::before_transmit() callback may help.

If you have the source code for the master then we might be able to debug the problem from both ends.

djholt commented 1 year ago

Thanks again, Richard. I've confirmed a few things now and have good news.

The master microcontroller is the Microchip PIC16LF1717 with hardware I2C and clock stretching support.

Writing to a different GPIO output that is not connected to the master does indeed still cause the stuck bus issue. Based on this I'm somewhat confident the problem is not with the master.

Disabling clock stretching on the Teensy does not seem to have any noticeable effect, nor does the change on line 453.

However, enabling the glitch filter on line 458 (together with the change on line 470) solves the issue reliably (so far).

I'll report back with any new information, but I'm hopeful the problem is solved now. Again, I very much appreciate your support and quick turnaround with this. Thank you!!

Richard-Gemmell commented 1 year ago

Hi DJ, Thanks very much for giving me the results of your tests. That's really useful.

I guess writing to the GPIO is putting noise on the I2C bus. The slave then mis-reads as a signal and gets out of step with the master.

I tried reproducing with a Teensy on it's own but couldn't get it to happen. :(

Please can you try the glitch filter settings on line 459 and let me know if that fixes it too.

cheers, Richard

djholt commented 1 year ago

Yes, the glitch filter setting on line 459 works just as well to solve the issue as the setting on line 458.

Thank you again!

Best, DJ

Richard-Gemmell commented 1 year ago

Thanks for that.

Good luck with your project. Richard