Combine Soundreactive 2D and FastLED_2D implementations

[embedded-creations]

WLED Soundreactive has a 2D implementation based around a single matrix where multiple segments can be mapped onto the matrix, and multiple panels can be stacked horizontally and vertically to create a larger matrix. My FastLED_2D branch has a 2D implementation based around pixel coordinate mapping, with support for multiple segments, each with its own 2D map. Several of us want to combine our 2D efforts and make a 2D implementation that supports all of the above and more, that can be integrated into the main WLED repo.

Right now this issue is just a place to discuss the idea and collect requirements:

Must

• All the effects in the SR repo run as they do now (or better) with the new API
• Support @Harry Baas' created presets using the existing effects in ‘spectacular’ combinations (like the overlapping segments) which should keep working as well

Want

Like

Consider

• Support Framebuffer_GFX so other 2D libraries like Adafruit_GFX can be used to draw to segments?
• ledmap.json can only handle 800 or so LEDs, maybe we need to ensure that 1D LEDs set to low bus indices, before 2D LEDs
• Support polar (or other types of) coordinates by allowing the effect to specify the coordinates it wants, and updating the mapping when the effect starts?
• Support easily changing up segment mapping at runtime, like you do with palettes? Examples:
  • Change the mapping upside down so a fire effect now looks like lava?
  • Multiple matrices mapped in a "scene" each with a unique non-overlapping position in XY space for each matrix, to all matrices now overlapping and centered so they play the same content.
  • "Zoom" in and out by scaling to change how the effect looks
• "Upgrade" 1D strip to 1 pixel high 2D strip when used in 2D effects?
  • Can apply transformations on a 1D segment and get X/Y coordinates without actually making a map. Use a reasonable default transformation if not set.
  • Think about how Pixelblaze does Render1D for 2D/3D effects

Issues

• Segments UI needs to be modified to add 2D parameters, and remove 1D parameters, this is likely something that needs to be integrated into main WLED or maintained in a fork
• LED Settings needs to be modified to add mapping config, something that can't currently be done without forking WLED
  • Blaz is already prototyping an improved Usermod Settings improvement that may work

[ewoudwijma]

One addition: WLED SR 2D supports multiple matrix panels which can be stacked horizontally and vertically to create a larger matrix

[embedded-creations]

Thanks, I edited the description

[embedded-creations]

I've been editing the description capturing important notes from our discussions in the Discord #2d channel, and I'll keep doing that.

[hi5grant]

I’m very interested in this feature as a user. Some of my questions and brainstorming will be more out of ignorance of the software so just know that.

• Do you envision the first iteration to be preset animations? Maybe a handful of them to start?
• That brings up another question, do the animations need to be exact to the pixel resolution? User matrix dimensions are going to vary. Would there need to be handling to shrink or pick a subsection of the display?
• Any user-created, or user-supplied videos or GIFs as content that can be rendered into the matrix? I don’t imagine the hardware storage could support that; maybe the native apps or web page could shrink the resolutions and adjust frame rate?

[embedded-creations]

@hi5grant Join the WLED Discord, there's a lot going on in the #2d channel. https://discord.gg/e2NW9hwPeV

Ideally effects should scale to the user's hardware, some will handle that better than others. Content like GIFs won't scale easily on device, but the content can be scaled before loading. I have GIF playback from the onboard storage or external SD card working already on my branch.

[softhack007]

@ewoudwijma, @embedded-creations, @blazoncek: is this topic still open for WLED 0.14.0 ? If yes, what are the points that need improvements or further attention?

In case nothing left to do, I suggest we close this one, and maybe open a new feature request for things still to be done in 0.14.x versions.

[blazoncek]

I have nothing to do with it, but I implemented my own 2D functionality and ported useful and relevant SR effects upstream (with your and @ewoudwijma help) while preparing 0.14. 😄 I am ok to close it.

[softhack007]

😃 yes, that was a huge joint effort 👍 Let's close this one now.

@ewoudwijma @embedded-creations not sure about the framebuffer-GFX part... please open a new feature request if you like.

[ewoudwijma]

I think it is still good to compare the design of @embedded-creations with what we have now as we have a lot of mapping systems, see https://mm.kno.wled.ge/moonmodules/mappings/ where I try to list the mappings, a description should follow (wip). Maybe I add more description in the weekend.

@embedded-creations , what do you think, what is the current status of your mapping system and how does it compare to what we have now and could we benefit from it?

[marcmerlin]

Since I can't re-open the original bug, I was prodded by others who use wled and told me it's more ready for 2d support today, to re-open a bug on the mapping issue, so I did do here https://github.com/Aircoookie/WLED/issues/3906

I'll let you decide what route you think is best, but obviously adding neomatrix mapping support is kind of important if you want to be able to support 2D matrices