Random LEDs turn on to random colors when state composer running

[CarvellWakeman]

Steps to reproduce: 1) Set up a minimum usable system with fewer LEDs than exist on the LED strip 1.5) This system should have a single LED state, single daily state, single profile, single zone, one controller, and < 60 LEDs 2) Set this system as the current profile and confirm that the LEDs light up.

Expected: Only the LEDs added to the system light up to their appropriate color, and remain that color indefinitely.

Actual: After some period of time, usually short, random LEDs will turn on to random colors. This does not include the LEDs added to the system. The state composer is sending data constantly so those LEDs are always being set to the correct color, but other LEDs are being sent data for some reason.

[cascadeth]

2 things to implement:

1. If error on serial write, send again

2. Create temp state object to check against for composer (get rid of constantly updating/sending)

[cascadeth]

on error, serial_send() will now attempt to send the I2C frame two more times, and give up after

[cascadeth]

Nano's have now been seen loosing sync with the I2C bus clock produced by the Pi.
This will create a forever broke nano, and error the rest of the bus. An entire strip of LEDs may be effected.

Proposed solution: Along with only updating the LEDs when the composer detects a zone LED state change (build unordered mapping), only open the I2C bus file stream to write data to the nanos, on this detect.

Note: The nanos may not like the change (starting and restarting the I2C clock, may throw off the nanos expected constant I2C clock). The opening and closing of the file stream may also hinder performance of the Composer.

[CarvellWakeman]

State composer only sends state when it changes, and also opens the I2CFileStream right before writing, and closes it afterwards. This seems to be working properly, but more long term testing is needed.

[CarvellWakeman]

I have done an overnight test changing state every 30 minutes. This morning it was exactly how I expected it to be. I have since seen it change 3 times and every time there were no LED errors. This seems to have done it, but longer term testing is needed. I will probably leave it running for the rest of the day.

[cascadeth]

I too have run through different tests. Few LEDs on a strip have been blipped randomly. Swapping the strip to a different nano, the nano seems to be the issue. I may remove this nano, and swap in a new nano.

[cascadeth]

As per discussion with @CarvellWakeman, we need to start looking at what kind of values the nanos are getting. It may be during a single blip, which is repaired by the composer getting another try at the serial_send(), that other nanos may also receive a bad command on the I2C bus. We are thus going to look into the nano code as well to check for receiving errors. If the composer is unable to write, the nanos may not fully receive the entire set of data for an LED.

[cascadeth]

Created a new nano code set and started testing values we get from the composer. Everything is as it should be. However we are still seeing blips. Watching the behavior of the nanos:

• One will pause when receiving I2C data, and continue.
• When static and not receiving commands, nanos will lose sync (I woke up to incomplete color strips) and stop receiving I2C data

Only a full reset of the nano seems to get it back online with I2C. What is also funny is when a nano stops receiving I2C transmissions, the composer is completely unaware, since it does not fail to write to the I2C address in question.

[cascadeth]

New test to see if the Composer and serial_send are the issue (maybe): leaving the nanos on a particular color. No composer or control service running. If things change, we will have a better answer.

[cascadeth]

With the explicitly defined (faster) baud rate, I think it's safe to say this issue is resolved. We have seen the system run for hours (expo) without any further blips. Closing the issue; can be reopened if needed.