WS2811 w/ 600 LEDs is not controllable via Arduino Mega

[zaneclaes]

I successfully tested my setup using an Arduino Uno. It was connected to hyperion on a Raspberry Pi 4 (via USB), but was limited to ~400 LEDs due to memory constraints. I therefore swapped in a Mega so that I could run all 600 LEDs in my chain of WS2811s. When I reset the Mega, I do correctly see it blank the lights (which I had enabled in the defines), so I believe wiring and such is correct. I also see the Ada prompts in the Serial Monitor. However, when I attempt to control the Mega via Hyperion / Raspberry Pi, nothing happens (if there's a simpler or more narrowly targeted way to test, I'm all ears).

I have verbose logging turned on in Hyperion, and here's what I see:

Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.821 hyperiond LEDDEVICE    : <DEBUG> LedDevice.cpp:148:init() | deviceConfig: [{"colorOrder":"rgb","currentLedCount":600,"del
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.821 hyperiond LEDDEVICE    : <DEBUG> LedDevice.cpp:407:setLatchTime() | LatchTime updated to 30ms
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.822 hyperiond LEDDEVICE    : <DEBUG> LedDevice.cpp:428:setRewriteTime() | Refresh interval = 1000ms
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.822 hyperiond LEDDEVICE    : <DEBUG> LedDevice.cpp:434:setRewriteTime() | RewriteTime updated to 1000ms
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.822 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:36:init() | DeviceType   : adalight
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.822 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:37:init() | LedCount     : 600
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.822 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:38:init() | ColorOrder   : rgb
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.823 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:39:init() | RefreshTime  : 1000
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.823 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:40:init() | LatchTime    : 30
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.823 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:52:init() | deviceName   : auto
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.823 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:53:init() | AutoDevice   : 1
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.823 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:54:init() | baudRate_Hz  : 115200
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.823 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:55:init() | delayAfCon ms: 0
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.823 hyperiond LEDDEVICE    : <DEBUG> LedDeviceAdalight.cpp:59:init() | Adalight header for 600 leds: Ada 0x02 0x57 0x00
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.874 hyperiond LEDDEVICE    : <INFO> found serial device: ttyUSB0
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.874 hyperiond LEDDEVICE    : <INFO> Opening UART: ttyUSB0
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.874 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:141:tryOpen() | _rs232Port.open(QIODevice::ReadWrite): ttyUSB0, Baud r
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.918 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:146:tryOpen() | portName:          ttyUSB0
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.918 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:147:tryOpen() | systemLocation:    /dev/ttyUSB0
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.918 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:148:tryOpen() | description:       USB2.0-Serial
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.918 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:149:tryOpen() | manufacturer:      1a86
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.918 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:150:tryOpen() | productIdentifier: 0x7523
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.918 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:151:tryOpen() | vendorIdentifier:  0x1a86
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.918 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:152:tryOpen() | serialNumber:
Mar 04 18:08:49 deck hyperiond[600]: 2021-03-04T18:08:49.926 hyperiond COMPONENTREG : <DEBUG> ComponentRegister.cpp:36:setNewComponentState() | LED device: enabled

The only change I made to the .ino between the Uno and Mega was to change the number of LEDs.

It looks like Hyperion recognizes the USB and is sending messages, but those messages are not being parsed(?). Is there some internal limit I'm hitting? I know FastLED can support thousands of WS28xx... and the Mega has plenty of memory for this (8x that of the Uno, which could almost fit all 600 LEDs).

[dmadison]

Hi Zane. Off the top of my head I can't think of a reason why it shouldn't work. In theory the code should support thousands of LEDs, although I've never tested that myself in hardware.

I'd start by testing the LEDs themselves with one of the FastLED examples. Are all 600 working like you expect them to? That should rule out wiring and power issues.

From there and in no particular order:

• Try some different software like Prismatik. Do all 600 LEDs work there? In that case it may be a Hyperion issue.
• Modify the number of LEDs in the firmware. At what number does it start working? At what number does it stop working? This could indicate a memory problem.
• Do the LEDs respond when the driver software is sending fewer LEDs? If yes, when does the break occur? That could point to a serial or parsing error in the firmware.

[zaneclaes]

Hm. I know all 600 lights in the strand work, FWIW. It's the Mega which appears to be the problem (doesn't work with 400 LEDs like the Uno does). I'm tempted to blame the Mega hardware itself, but it had been used on a prior project with no known problems.

Do you happen to know where I can find a brief explanation of the Adalight protocol? Specifically, I want to manually send valid instruction lines via the serial monitor in the Arduino IDE, so I can test in the most direct way possible.

[dmadison]

Even if you've tested the LEDs with the Uno I'd run one of the FastLED examples on the Mega just to rule out any issues with the non-Adalight side of things.

The best explanation of the Adalight protocol is probably in the source code itself. Here's the comment:

// A 'magic word' (along with LED count & checksum) precedes each block // of LED data; this assists the microcontroller in syncing up with the // host-side software and properly issuing the latch (host I/O is // likely buffered, making usleep() unreliable for latch). You may see // an initial glitchy frame or two until the two come into alignment. // The magic word can be whatever sequence you like, but each character // should be unique, and frequent pixel values like 0 and 255 are // avoided -- fewer false positives. The host software will need to // generate a compatible header: immediately following the magic word // are three bytes: a 16-bit count of the number of LEDs (high byte // first) followed by a simple checksum value (high byte XOR low byte // XOR 0x55). LED data follows, 3 bytes per LED, in order R, G, B, // where 0 = off and 255 = max brightness.

The magic word is 'Ada', followed by the LED count and checksum, followed by the color data in order. Note that the LED count is indexed at 0 (so number of LEDs - 1).

For example a header for 80 LEDs would be: A (0x41) (magic word) d (0x64) (magic word) a (0x61) (magic word) 0 (0x00) (high byte, LED count - 1) 79 (0x4F) (low byte, LED count - 1) 26 (0x1A) (checksum, hi ^ low^ 0x55)

[zaneclaes]

Well, I'm still not at the bottom of this. It's rather odd...

Here's what I did, step by step, after making a fresh clone of the repo:

• Enable DEBUG_LED and CLEAR_ON_START
• Set Num_Leds to 100 and use WS2811.
• Install on an Arduino Uno. Verify it works with Hyperion as-expected.
• Install on an Arduino Mega 2560 R3. Again, it clears the LEDs during boot, but never takes commands.
• I then tried installing a different FastLED sketch onto the same Mega. It worked without any problems.
• I purchased a second Mega and replicated all of the above results.

At this point, I started fiddling with the debug lights. I can confirm that the checksum is being met via the debug light in the latch block. If I turn the debug light off at the very first line of dataMode(), it still appears to stay on constantly. I have disabled the serial timeout, just in case, to limit the possible causes of mode switching. I even tried adding a delay(100) after the LED gets turned off, but I never see it blink off.

I then tried manually sending a packet:

echo -en '\x41\x64\x61\x00\x4F\x1A' > /dev/cu.usbmodem22301

I see the same pattern of debug lights on both Mega and Uno. This suggests that the sketch is fine. But I don't understand why the message received from Hyperion would not be understood. Especially because the boards are running the same code, and I'm not changing any Hyperion settings (the port is auto-detected, and I reboot just in case).

My best hypothesis is some sort of miscalculation of bytesRemaining. It almost smells like an underflow or something...

[dmadison]

Just to make sure I'm following you here: the vanilla sketch with 100 LEDs works fine on the Uno, but doesn't work on the Mega?

[zaneclaes]

Just to make sure I'm following you here: the vanilla sketch with 100 LEDs works fine on the Uno, but doesn't work on the Mega?

Precisely, assuming you consider the first two bullet points from the last post to still constitute "vanilla."

[dmadison]

Indulge me for a minute: does the Mega work if you set the number of LEDs to 80? Both in the code and in Hyperion.

[zaneclaes]

Here is the exact code I just uploaded to the Mega (with 80 LEDs). The moment Hyperion connects, the Debug light turns on and the RX light flickers (not quite constant, but close).

The fact that the Debug + RX lights are both "pegged" in the "on" state is highly suspicious to me. Nothing seems to release it from this state, notwithstanding adding a D_LED(OFF) to the end of loop(). And since RX is pegged, I'm left to suspect that the serial is stuck in a "read" state.

I wish I had the exact message format from Hyperion. But again, since everything works with the Uno, I worry that is not a productive line of inquiry.

[zaneclaes]

FWIW, the UNO running the same sketch does show a flashing debug light when receiving from Hyperion, as expected.

edit: I was wondering if the Mega was just too fast to notice, but I'm under the impression both have the same clock speed. Since adding a delay(100) to the "end of message" (mode = Header) block also did nothing, I don't think the message is ever getting processed.

Here are Hyperion screenshots:

[dmadison]

That's definitely bizarre. I could be wrong but this screams "memory issue" to me. Try commenting out the memset calls on lines 195 and 230.

[zaneclaes]

Yeah, I was thinking some kind of under/over-flow — if not the byteCount, then the memset. FWIW, I had previously tried moving the D_LED(ON) to occur after the memset, so at least we know the call executes without any kind of deadlock. I did commented out both memsets now and am not seeing much difference, though I think it may have turned the debug light off after ~5 minutes. I just restarted Hyperion again to see if it does so again.

[zaneclaes]

The plot thickens...

I finally went back and just installed a simple Blink sketch on the Mega, then connected it to Hyperion. The symptoms are the same, i.e., the L light stays locked on and the RX light flickers rapidly. I notice that if I set the number of (Hyperion) LEDs to be high, then the RX light stays on constantly, while if it is low, the RX light blinks more intermittently. Yet since the Debug light is not blinking, it almost seems like the CPU is pegged trying to receive data.

Whatever the problem is, it does seem outside this repo, but I'm curious if you have any insight. Consider even this very basic sketch. When it starts, the TX light is solid (and nothing else). As soon as Hyperion connects, the L goes solid and the TX turns off (the RX starts flickering).

#include <Arduino.h>

void setup() {
  // put your setup code here, to run once:
  pinMode(13, OUTPUT);
  digitalWrite(13, LOW);
  Serial.begin(115200);
}

void loop() {
  Serial.println("TEST");
}

I notice that Hyperion is auto-detecting the latch time, refresh interval, etc. They appear to be the same for Uno and Mega, but I wonder if there's some reason these won't work for the Mega...

CE    : <DEBUG> LedDevice.cpp:148:init() | deviceConfig: [{"colorOrder":"grb","currentLedCount":8,"delayAfterConnect":0,"hardwareLedCount":1,"latchTime":30,"lightberry_apa102_mode":false,"output":"auto","rate":115200,"rewriteTime":1000,"type":"adalight"}]
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.443 hyperiond LEDDEVICE    : <DEBUG> LedDevice.cpp:407:setLatchTime() | LatchTime updated to 30ms
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.443 hyperiond LEDDEVICE    : <DEBUG> LedDevice.cpp:428:setRewriteTime() | Refresh interval = 1000ms
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.443 hyperiond LEDDEVICE    : <DEBUG> LedDevice.cpp:434:setRewriteTime() | RewriteTime updated to 1000ms
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.443 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:36:init() | DeviceType   : adalight
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.444 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:37:init() | LedCount     : 8
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.444 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:38:init() | ColorOrder   : grb
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.444 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:39:init() | RefreshTime  : 1000
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.444 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:40:init() | LatchTime    : 30
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.444 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:52:init() | deviceName   : auto
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.445 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:53:init() | AutoDevice   : 1
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.445 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:54:init() | baudRate_Hz  : 115200
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.445 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:55:init() | delayAfCon ms: 0
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.445 hyperiond LEDDEVICE    : <DEBUG> LedDeviceAdalight.cpp:59:init() | Adalight header for 8 leds: Ada 0x00 0x07 0x52
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.524 hyperiond LEDDEVICE    : <INFO> found serial device: ttyACM0
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.524 hyperiond LEDDEVICE    : <INFO> Opening UART: ttyACM0
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.524 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:141:tryOpen() | _rs232Port.open(QIODevice::ReadWrite): ttyACM0, Baud rate [115200]bps
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.588 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:146:tryOpen() | portName:          ttyACM0
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.588 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:147:tryOpen() | systemLocation:    /dev/ttyACM0
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.589 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:148:tryOpen() | description:       0042
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.589 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:149:tryOpen() | manufacturer:      Arduino  www.arduino.cc
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.589 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:150:tryOpen() | productIdentifier: 0x42
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.589 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:151:tryOpen() | vendorIdentifier:  0x2341
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.589 hyperiond LEDDEVICE    : <DEBUG> ProviderRs232.cpp:152:tryOpen() | serialNumber:      75932313938351B02151
Mar 10 14:05:52 deck hyperiond[3926]: 2021-03-10T14:05:52.592 hyperiond COMPONENTREG : <DEBUG> ComponentRegister.cpp:36:setNewComponentState() | LED device: enabled

[dmadison]

Have you tried testing with other Adalight-compatible software such as Prismatik? That would eliminate Hyperion as a factor.

[zaneclaes]

I did, though having never used it before I found the UI confusing and I was not certain that I had actually done things correctly (I couldn't find any connection status indicator or other way to determine what was going on). Now that I understand the sketch (and its use of debug lights) better, I tried Prismatik again... maybe it's not well-tested on a Mac, but it's really... not working well. I suspect it has to do with permissions in later versions of Mac OS, as many buttons simply don't do anything at all.

[zaneclaes]

Success! The key was to add a startup delay before sending Ada messages to the Mega. In Hyperion, this can be done via the delayAfterConnect setting (which is not exposed in the UI, so I had to manually edit the .sqlite database).

[dmadison]

Interesting. I'm glad you figured it out, and thanks for sharing the solution!