Enhance Sound-Reactive Functionality with musical analysis (Colorchord, Pianolizer, etc)

[Fireflaker]

Is your feature request related to a problem? Please describe. Although WLED already has built-in support for sound-reactive lighting through the Sound-Reactive-WLED fork, there may be opportunities to improve and expand upon this functionality by integrating the Colorchord embedded system. Colorchord is a chromatic sound-to-light conversion system that looks for notes in the audio input and produces corresponding light patterns, creating more interesting patterns between instruments and music. Implementing it on ESP32 will also allow support of more than approx 250 pixels, as well as the support of much more advanced LED driver libraries.

Describe the solution you'd like Integrate Colorchord embedded system into WLED to enhance its sound-reactive lighting capabilities. Colorchord embedded is designed for 32-bit systems such as STM32F303, STM32F407, and ESP8266.

Describe alternatives you've considered N/A The idea of folding 880 on top of 440Hz is rather novel and creates effective audio visualization.

Additional context Colorchord GitHub Repository: https://github.com/cnlohr/colorchord/tree/master/embeddedcommon

Colorchord provides a unique approach to sound-reactive lighting by focusing on the chromatic properties of sound rather than frequency ranges. Integrating this system into WLED could offer more engaging and responsive lighting patterns for users.

[softhack007]

Interesting link 👍 Technicially it seems to be similar to "pianolizer" https://github.com/creaktive/pianolizer

[Fireflaker]

Thank you for the modifications! Just a few interesting things to point out:

• • Colorchord runs on ESP8266 at ~125hz while Pianolizer requires Raspberry Pi

• ColorChord's DFT (Discrete Fourier Transform) implementation is based on a custom algorithm that uses an optimized sliding DFT, which is a modification of the Goertzel algorithm. This approach allows ColorChord to detect chromatic properties of the sound in real-time with lower computational complexity compared to a traditional DFT.

• The core of ColorChord's DFT implementation can be found in the embeddedcommon folder within the ColorChord repository. The main functions related to DFT are in the DFT32.c file.

• The effect of Colorchord on a 1d string of ws2812 looks drastically different from any other FFT visualizer I have seen.

[softhack007]

• while Pianolizer requires Raspberry Pi

Well I may know more ... I know for sure that the author of Pianolizer is currently porting his core algo to esp32, so it may still be an I interesting option to consider.

Also there are other DSP libraries that might be interesting in this context:

• aubio is a very feature rich lib for audio analysis, core is written in c/c++ https://github.com/aubio/aubio
• esp-dsp is a package from espressif, with some extremely fast implementations of FFT and general-purpose filters: https://github.com/espressif/esp-dsp
• esp-adf and esp-sr could be used to bring voice control into wled, however harder to integrate https://github.com/espressif/esp-sr, https://github.com/espressif/esp-adf

So I'd like to keep this topic a bit wider than colorcrome only.

[Fireflaker]

CLosing because the Freqwave function works well enough as a good substitute for Colorchord with the following settings.

[softhack007]

Nice to hear that we already have something to produce similar effects 👍 thanks for sharing 🥇

I still like to keep this open and come back to it later. Advanced musical analysis is an interesting topic for future enhancements, no matter which implementation we might chose.

[softhack007]

Adding another tool to the list that includes "chromatic analysis" : https://github.com/connornishijima/SensoryBridge