search

Aircoookie / WLED

Control WS2812B and many more types of digital RGB LEDs with an ESP8266 or ESP32 over WiFi!
https://kno.wled.ge
MIT License
14.29k stars 3.05k forks source link

OpenCV (?) led mapping #1628

Open abyssdj opened 3 years ago

abyssdj commented 3 years ago

Bear with me on this one as I'm not sure how feasible it would be to implement this with WLED, but it would be great to have an integrated method of mapping a "mess" of led's. For example, taking the 20m string of tiny ws2812b's that I've just got on amazon, throwing them all on the floor, and yet still using them as a matrix-esque arrangement. I'm not explaining this very well, take a look here - https://github.com/PWRFLcreative/Lightwork

I had a quick look at the software yesterday but couldn't get it to run in processing... could it be done? I can see a lot of uses for it!

Legsmaniac commented 3 years ago

It looks/sounds similar to Twinkly, the Christmas Tree lights.

https://www.twinkly.com/

abyssdj commented 3 years ago

It looks/sounds similar to Twinkly, the Christmas Tree lights.

https://www.twinkly.com/

Yes, that does look like basically the same concept. Have you seen the prices?!

Legsmaniac commented 3 years ago

Have you seen the prices?!

I actually bought one in a sale last year but I wasn't that impressed with it, I returned it for refund. 😂

abyssdj commented 3 years ago

Have you seen the prices?!

I actually bought one in a sale last year but I wasn't that impressed with it, I returned it for refund. 😂

Wow, that bad? Even more call for an open source option then, right?!

Keithpohl82 commented 3 years ago

I believe XLights does this with its custom models setup. If i'm not mistaken, you record a short video of each light in a string lighting up and it maps it to an excel type sheet . It has some bugs but effective. Similar could be done with WLED but I think the phone app would be limited

abyssdj commented 3 years ago

I'll look into that. Not bothered personally about a phone app, as everything would be running from artnet/sacn, maybe what it actually needs in my case is software on the computer

Keithpohl82 commented 3 years ago

I'll look into that. Not bothered personally about a phone app, as everything would be running from artnet/sacn, maybe what it actually needs in my case is software on the computer

Web API should be able to handle it. Its just on how you would get the video on to a computer for most. Video could be recorded on phone and used in the phone app. The only thing with this would be it would have to be "Remapped" any time the lights moved. If the lights dont move or changed positions it would be no problems.

Aircoookie commented 3 years ago

I am definitely interested in implementing this feature natively in WLED at some point! It just makes for amazing effects! Need to put some development into supporting something OpenCV-ish in the app, as a phone is usually the most accessible camera enabled device. I'm afraid with a webcam and standalone software it would get a lot less use, but it is a great starting point. Javascript/browser native is out of the question, as it doesn't allow camera access without HTTPS. Mapping should also be easily exportable to use in xLights or other tools, not only for built-in effect :)

Legsmaniac commented 3 years ago

Even more call for an open source option then, right?!

I would certainly be interesting in giving it a go. I plan to make my own Christmas tree this year ready for next Christmas, with built in lights. This would certainly be an option. 😉

abyssdj commented 3 years ago

I am definitely interested in implementing this feature natively in WLED at some point! It just makes for amazing effects! Need to put some development into supporting something OpenCV-ish in the app, as a phone is usually the most accessible camera enabled device. I'm afraid with a webcam and standalone software it would get a lot less use, but it is a great starting point. Javascript/browser native is out of the question, as it doesn't allow camera access without HTTPS. Mapping should also be easily exportable to use in xLights or other tools, not only for built-in effect :)

Yeah, I think it would open up a lot of very creative options even just with the built in effects. If it can be made to work relatively pain free with artnet etc, even better! Almost like a "scruffy matrix", rather than everything being perfectly aligned and clinical.

I was thinking, maybe an option would be an extra tab in the app which basically plays a sequence, records a video and then exports to a csv (or whatever), which could then be uploaded to WLED in a similar manner to uploading firmware or as you say, imported into xlights, ELM, whatever?

baerni commented 2 years ago

Hi. Is there any progress on the topic?

blazoncek commented 2 years ago

No.

SenMorgan commented 9 months ago

That guy made opensource Twinkly copy, with Android app and Arduino sketch: https://github.com/AlexGyver/GyverTwink

blazoncek commented 9 months ago

Now we need a volunteer to port it to WLED. 😉

lypoluz commented 7 months ago

Is there a way to manually map leds to certain positions yet? I've seen the option to provide a gap file for the 2D setup. But that wouldn't solve the issue of randomly placed leds, or in my case, my matrix is layed out like a snake.

blazoncek commented 7 months ago

Is there a way to manually map leds to certain positions yet?

For at least 4 years if not more. 😉 KB

roboraptor commented 7 months ago

Ba Bump Tss, i guess we still didnt have any app to map right?

BjarneJensen commented 1 month ago

WLED's remap only takes led index into account afaik, so I'm not sure how one would use the tracking info beyond sorting and reordering them. Maybe that's enough in some cases, but I imagine a more spatial integration where one could set (matrix) effects' resolution/aspect to match the camera and led color would be defined by "sampling" the effect at the uv(xy) coordinates of leds as defined the camera and some calibration effect.

These may be of interest in addition to GyverTwink: https://github.com/atomic14/self-organising-leds esp32 automap leds https://led3dmap.lights0123.com/ Generate 3d map from 2 videos and FastLED calibration pattern. Web interface for defining number of lights and duration (might be as simple as adding the calibration effect as to WLED and recording a video)

blazoncek commented 1 month ago

There's also this #3903 and this #4018

BjarneJensen commented 1 month ago

There's also this #3903 and this #4018

Thanks. I ended up making a matrix of 32x24 to match my camera aspect, then created 2dgaps and ledmap files based on uv coordinates of 50 leds. It actually matched up pretty well. Not sure the memory implications of drawing a grid that big only to light 50 leds. Cant be optimal, but concept works.

led1 led