Improve I2C recovery attempt

[ShadowLight8]

When I2C is having errors, an attempt is made to restart the transmitter:

2022-10-23 09:24:10,654:root:ERROR:read_i2c_block_data error
Traceback (most recent call last):
  File "/home/fpp/media/plugins/Dynamic_RDS/Dynamic_RDS_Engine.py", line 78, in read
    retVal = self.bus.read_i2c_block_data(self.address, address, num_bytes)
OSError: [Errno 121] Remote I/O error
2022-10-23 09:24:10,779:root:ERROR:rdsSentStatusToggleBit failed to flip
2022-10-23 09:24:10,780:root:INFO:Stopping QN80xx transmitter
2022-10-23 09:24:10,780:root:DEBUG:Stopping PWM
2022-10-23 09:24:10,781:root:INFO:Disabling PWM
2022-10-23 09:24:11,783:root:INFO:Starting QN80xx transmitter
2022-10-23 09:24:12,187:root:DEBUG:Setting up PWM
2022-10-23 09:24:12,188:root:DEBUG:Setting PWM period to 18300
2022-10-23 09:24:12,189:root:DEBUG:Setting PWM duty cycle to 1525
2022-10-23 09:24:12,190:root:INFO:Enabling PWM
2022-10-23 09:24:12,368:root:ERROR:read_i2c_block_data error
Traceback (most recent call last):
  File "/home/fpp/media/plugins/Dynamic_RDS/Dynamic_RDS_Engine.py", line 78, in read
    retVal = self.bus.read_i2c_block_data(self.address, address, num_bytes)
OSError: [Errno 121] Remote I/O error

Might also be worth reinitializing the smbus object.

[ShadowLight8]

Initial testing is looking good from the functional view of this, however I've not seen a true recovery yet.

[ShadowLight8]

Discovered that the PWM wire in parallel to the I2C wire will cause interference with I2C and many of these issues are caused by that. However, now it allows for more robust testing, which has been good so far.

[TomasRa1]

A lot of this and similar I/O errors appeared in case when the transmitted high-frequency RF signal goes from the transmitter module (or from its improperly matched antenna) to the RPi module via the connecting cable or radiated from antenna via the air. RPi has very low EMI resistance and needs to be shielded. It will also help a bit to thread a ferrite EMI two-piece core on the connecting cable. I tested small Chinese with no effect and "3M" with noticeable improvement. Next, I plan to test the SOIC-8 ADUM1250ARZ bidirectional galvanic separator i2s SCL and SDA signal. Now I am waiting for IC and PCB from China. Furthermore, I also performed a series of measurements on the i2c bus with an oscilloscope using the original Chinese control processor supplied with the transmitter module, whose i2c communication was always reliable without I/O errors at all cases. I also did a lot of measurements with the TX-module controlled by the RPi with a lot of i2c I/O errors. I consulted the results at: https://github.com/raspberrypi/linux/issues/4884#issuecomment-1778549161 with Mr. Roger Wolff. He advised me to try i2c communication via "SOFTWARE I2C" https://learn.adafruit.com/raspberry-pi-i2c-clock-stretching-fixes/software-i2c I would like to try replace i2c communication via SMBus by "SOFTWARE I2C" for more reliable communication with try to decrease (or eliminate) I/O errors. This would mean modifying the Python code of the Dynamic_RDS_Engine.py file. I'm not sure if my proposed solution would work so I want to ask the author Mr. Nick Anderson and ask for correction. There are basic modification only for "def.write" (the same it needs also for "def.read").

/boot/config.txt add: dtoverlay=i2c-gpio,i2c_gpio_sda=2,i2c_gpio_scl=3,i2c_gpio_delay_us=4,bus=4 Then it would see in $ **ls /dev/i2c* : /dev/i2c-1 /dev/i2c-4 Then it must install to Rpi3B+: pip3 install adafruit-extended-bus** Then add and change in Dynamic_RDS_Engine.py file:

# import the library (replace "import smbus" by:)
from adafruit_extended_bus import ExtendedI2C as I2C

# inside py-code in init of i2c replace "self.bus = smbus.SMBus(bus=1)" by:
self.bus = I2C(4)

# and then send  under "def write(self, address, values, isFatal = False):" data via new way to i2c bus :
self.bus.write_i2c_block_data(self.address, address, values)

Nick, please check and correct it for to be working and I will test it. Thank you very much! Tomas

[ShadowLight8]

Thanks for the info @TomasRa1 - I'll start looking into it. I could see this being an option when the interference is causing the regular i2c bus to have too many issues.

[ShadowLight8]

I got side tracked with the Engine restart loop issue. I'm planning to dig into this more next.

[TomasRa1]

According to the records from the oscilloscope, it seems that the error rate of i2c communication when generating SCL and SDA by the SMBUS of RPi processor is causing:

1. bad timing synchronization between SCL and SDA - the leading edges of the pulses are too close to each other, compared to the signal from the original Chinese CPU, where the edges of the pulses, i.e. the log.1 and log.0 state changes, are further apart.
2. The RPi sometimes generates a very short pulse (around 1usec.) on SDA, which the original Chinese CPU never generates.

Both these problems should be solved by the use of "SOFTWARE I2C" instead SMBUS for generating the SCL and SDA signals.

First the missing pip (pip3) SW needs to be installed on FPP 7.3 and add set of system path:

sudo curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python3 get-pip.py
PATH=$PATH:/home/fpp/.local/bin

than uninstall SMBUS (from FPP) and reboot. than install SOFTWARE I2C :

pip3 install adafruit-extended-bus

Configure /boot/config.txt add: dtoverlay=i2c-gpio,i2c_gpio_sda=2,i2c_gpio_scl=3,i2c_gpio_delay_us=4,bus=4 and check /boot/config.txt that all others lines, which content i2c are commented by # sudo reboot and then test the existence of the new bus no.4 :

fpp@FPP:~ $ ls /dev/i2c*
/dev/i2c-4

then replace "import SMBUS" in Dynamic_RDS_Engine.py by: from adafruit_extended_bus import ExtendedI2C as I2C then please replace all SMBUS commands in Dynamic_RDS_Engine.py by: I2C(4)

Nick, thanks a lot for the perfect unicode-conversion implementation, great professional work! And thanks in advance for to try apply "Software I2C" in your plugin.

[ShadowLight8]

From Issue #25, I now have a consistent way to see I2C errors, so I'll work on setting up and testing out the software i2c option.

[ShadowLight8]

@TomasRa1 I found that if I commented out the hardware i2c being enabled in /boot/config.txt

#dtparam=i2c_arm=on

and added what you suggested, but setting the bus to 1

dtoverlay=i2c-gpio,i2c_gpio_sda=2,i2c_gpio_scl=3,i2c_gpio_delay_us=4,bus=1

it worked without having to make any code changes. I still need to test more, but so far it looks like this is a working option.

[TomasRa1]

i2c dual (SDA+SCL) bidirectional galvanic isolator - china started produce modul a few days ago: https://www.aliexpress.com/item/1005004881836766 If it will be also need for PWM signal, theoreticaly is possible to use cheaper variant of single bidirectional isolator: https://www.aliexpress.com/item/1005005344561371 I offered them and I will try to connect them between RPi and transmitter and test if reduce i2c errors in strong RF field. I would like to reach similar reliability as original china's CPU has (also for transmitter connected to RPi). For this state we also need replace "SMBUS" by "SOFTWARE I2C" in the "Dynamic_RDS_Engine.py" which could remove very short SDA spikes and should secure better time synchronisation/shift of SDA pulses (to SCL pulses) as we can see on pictures from osciloscope in https://github.com/ShadowLight8/Dynamic_RDS/issues/25

Interference from the PWM wire to SDA and SCL wires can be also improved by connecting a 22n ceramic capacitor to the transmitter board (next to the transmitter white connector or from below to the connector pins) between PWM pin and GND pin.

[TomasRa1]

@TomasRa1 I found that if I commented out the hardware i2c being enabled in /boot/config.txt

#dtparam=i2c_arm=on

and added what you suggested, but setting the bus to 1

dtoverlay=i2c-gpio,i2c_gpio_sda=2,i2c_gpio_scl=3,i2c_gpio_delay_us=4,bus=1

it worked without having to make any code changes. I still need to test more, but so far it looks like this is a working option.

I saw "Dynamic_RDS_Engine.py" code and now your solution with set bus=1 is clear for me. Thank you very much for this information. I will test it soon. Kind regards Tomas

[ShadowLight8]

@TomasRa1 Let me know how it goes! I'm working on a full solution where there will be a new check box to easily update the /boot/config.txt to use hardware or software i2c. You can see the work in progress on #27

[ShadowLight8]

@TomasRa1 I've merged #27 into main, so you should be able to update the plugin now and have the ability to switch between hardware and software i2c.

Please test with the software i2c and let me know if that works better.

[ShadowLight8]

Based on the testing feedback, the Software I2C option help improve things. I don't see any additional code changes needed for this, so I'm going to close it.