Prismatik HDR bug - Light blinking when moving mouse

[nhac-lly]

I have the issue here: https://www.youtube.com/watch?v=NgSb_JKcttU FYI the controller is CH340 chinese one.

[MegaVat30]

Same here. I have used ATmega328 CH340 Chinese replica and 2-3 days from installing everything was working normally, but suddenly the LEDs start flickering with different colors when I move the mouse. And it only happens when any window or app are running in fullscreen.

[psieg]

Is it related to HDR? Does it occur if HDR is disabled?

[nhac-lly]

@psieg Yes, it's HDR related since it's does not happened on SDR. My workaround is to have mouse trail enabled, I guess it's linked to refresh rate.

[sblantipodi]

I have the same problem here: https://github.com/psieg/Lightpack/issues/355

[sblantipodi]

the problem happen if your grab interval is "too fast" for what your PC can capture.

If you can capture at 30FPS, set the grab interval under 33ms, if you can capture at 60FPS, set the grab interval under 16ms...

this is not a solution but a workaround, it should work.

and it is another reason why I don't understand how nvidia can deprecate NVFBC. https://github.com/psieg/Lightpack/issues/235#issuecomment-639397507

[zomfg]

Try running this build with flashes and without. It should save frames in your %tmp%. Hopefully we'll be able to see what's going on. Save frames in between sessions as it'll override the previous ones (don't run long sessions, depending on your disk space)

[sblantipodi]

I see nothing strange in the image produced by your build zomfg.

Imho the problem isn't in the capturing step but in the output of the software from the PC to the microcontroller. Most probably it's not even a problem of the software that sends data to the microcontroller but simply in the transmission that isn't synced well.

I was able to reproduce the problem on a software written by my own.

the problem is that some "dirty transmission" isn't blocked by the checksum check you use before displaying the colors on the led strip. It's not a big problem thoug, it happen every three or four minutes and it last the time of a lightning.

[zomfg]

if by checksum you mean the 3 bytes after "Ada", that only applies to the led count, the color values aren't taken into account if you set logs to 2, that'll log the color values the are sent to the device, if you can capture it while running something like this, maybe we'll see some pattern...

[jespertheend]

I'm pretty sure this is unrelated to sending data to the microcontroller. I am using Prismatik without any external ledstrip connected to my computer. Instead I use the socket api that comes with Prismatik to send the color values to my own application, which then sends http requests to my lights. I'm seeing the same issue without any microcontroller for a ledstrip. In fact, if you look at the device tab, you can see the colors turning black sometimes.

Or at least, that's how it used to be for me anyway. I can't seem to replicate it anymore now, I recently switched from an amd card to one from nvidia, so perhaps that's related.

[sblantipodi]

Don't know what happened but I have the same problem of the opener. Prismatik is unusable when HDR is on

[MegaVat30]

In my case, the problem was solved by replacing all LED strips and wiring. Apparently I had some faulty LEDs.

[sblantipodi]

In my case, the problem was solved by replacing all LED strips and wiring. Apparently I had some faulty LEDs.

I don't think that this can be the case since it works ok without HDR

[sblantipodi]

Ok I finally release my implementation of Screen Capture for Adalight devices and I don't have any problems with HDR.

Obviously I don't post the link to my implementation because it would not be correct to you guys, I love your community and your work here. If someone is interested in having a working HDR can try my implementation on my GitHub.

I still remain a big fan of prismatik and hope that this problem will be solved in the future.

PS: Zomfg, sorry I had no time to do that try, I will do it as soon as I can.

[Benik3]

Hello. I'm also interested in HDR fix. I have now HDR 144Hz monitor. When switching to 10bit support or opening game with HDR I have this blinking.
I tried to play with the source codes. I successfully updated dxgi to use FLIP SWAP CHAIN. Then I tried to change DXGI_FORMAT to R16G16B16A16_FLOAT without success. With R10G10B10A2_UNORM I have no flickering, but I got weird colors on the LEDs :/ DXGI and C++ is absolutely new to me, so it's hard to try something :D

EDIT: Is interesting that it flash really only when actual FPS are lower then grab interval (or something like this) EDIT2: Ok it's not only about FPS. But it happens always when you move with mouse, but what is interesting if you grab e.g. window and move with window, it works ok

P.S. You can simulate this behaviour on SDR monitor by setting const DXGI_FORMAT DesktopFormats[] = { DXGI_FORMAT_R10G10B10A2_UNORM }; under DDUplGrabber.cpp

[zomfg]

interesting... could you push this to a branch on your fork? this issue is very confusing since the documentation says you'll always get 8bit data, so I assumed it would shift rgb30 to rgb24 before passing data... with HDR on, could you capture and save 1 raw frame with stock/8bit DDupl grabber, and 1 with your 10bit change with something like this (change file path) make half of your screen red/0xff0000 (fullscreen would be best), don't use a video source, make an image or html page maybe do a couple of other colors, just note the hex values thanks

diff --git a/Software/grab/GrabberBase.cpp b/Software/grab/GrabberBase.cpp
index 56a0b571..87278bf5 100644
--- a/Software/grab/GrabberBase.cpp
+++ b/Software/grab/GrabberBase.cpp
@@ -28,6 +28,7 @@
 #include "GrabberBase.hpp"
 #include "src/debug.h"
 #include <cmath>
+#include <QFile>

 namespace
 {
@@ -140,6 +141,7 @@ void GrabberBase::grab()
        ++grabScreensCount;
        _context->grabResult->clear();

+       const GrabbedScreen *gs = NULL;
        for (int i = 0; i < _context->grabWidgets->size(); ++i) {
            if (!_context->grabWidgets->at(i)->isAreaEnabled()) {
                _context->grabResult->append(qRgb(0,0,0));
@@ -149,6 +151,7 @@ void GrabberBase::grab()
            getValidRect(widgetRect);

            const GrabbedScreen *grabbedScreen = screenOfRect(widgetRect);
+           gs = grabbedScreen;
            if (grabbedScreen == NULL) {
                DEBUG_HIGH_LEVEL << Q_FUNC_INFO << " widget is out of screen " << Debug::toString(widgetRect);
                _context->grabResult->append(0);
@@ -222,7 +225,9 @@ void GrabberBase::grab()
                preparedRect);
            _context->grabResult->append(avgColor);
        }
-
+       QFile frameFile("/tmp/frame.dat");
+       frameFile.open(QIODevice::WriteOnly);
+       frameFile.write(QByteArray::fromRawData((const char*)gs->imgData, gs->imgDataSize));
    }
    emit frameGrabAttempted(_lastGrabResult);
 }

[Benik3]

I will try it.
Here you can find my updated repository:
https://github.com/Benik3/Lightpack/tree/HDR

EDIT: Hmm with later version of Lightpack I got this error in log. No problem with version 5.11.2.23 7aa0be41 13:00:45:079 Warning: void __cdecl SettingsWindow::updateVirtualLedsColors(const class QList<unsigned int> &) colors.count() 131 != m_labelsGrabbedColors.count() 110 . Cancel updating virtual colors. LedDeviceManager(0x1e77e1f2270) EDIT2: It looks like it use virtual LEDs even I have Ambilight in config. When I change num LED under Virtual in main.conf I don't get this error but LEDs doesn't work.

Anyway i found that the problem with color happens also if you change white balance in wizard.

[Benik3]

OK, I tried saving the frame as you mentioned. First interesting thing, it doesn't happen with colors with full value (e.g. R255 G0 B0, but also R255 G255 B0 ...).
With R10G10B10 on SDR the grab it see it like this: When I try full screen "dirty" yellow (R 0xFF G0xF2 B0), when LEDs are really yellow there is 00F2FFFF, but when the LEDs changes blinks to red, there is FF2F0FC0. It looks like some problem between RGB and BGR. When enabling 10bit in drivers, it see only FF2F0FC0. The same is with the blinking with using original B8G8R8.
Is true that normally Prismatik use B8G8R8, but 10bit and 16bit format is only in order R10G10B10 (R16G16B16).

P.S. I made another brach in my repository from older version of Prismatik, which works for me: https://github.com/Benik3/Lightpack/tree/dxgi%2BHDR_older

[Benik3]

I think that I found some trace.
Always when the LEDs blinks, it's when AcquireNextFrame returns DXGI_ERROR_WAIT_TIMEOUT. Otherwise I think that the 10bit DXGI_FORMAT is not the way, because when we use the 8bit, the dxgi should simply convert it and that's what we wont (LEDs are 8bit anyway)

[zomfg]

re-reading remarks here now makes more sense so according to this the current grabber should produce 8888 colors no matter what, and upgrading the API only allows for bypassing the 8888 conversion from whatever and yes, we don't want to convert 1010102 (or anything) ourselves because it's probably already very efficient as is but someone reported weird colors with 10bit mode iirc, so just to be sure it would be nice to look at what's captured you could alter my code snippet from above with something like this instead of QByteArray stuff

QString path = QStandardPaths::writableLocation(QStandardPaths::TempLocation);
QImage imog(gs->imgData, gs->bytesPerRow / 4, gs->imgDataSize / gs->bytesPerRow, QImage::Format_RGB32);
imog.save(QString("%1/priz_%2_og.png").arg(path).arg(grabScreensCount), "PNG");

try this with master build (so old API), and the new API with everything 8888, and check if images look weird I guess (but look closely, it might be subtle)

now why would AcquireNextFrame timeout all of a sudden while moving mouse?... try looking into #373, maybe it's related

[Benik3]

True, after re-reading the remarks I understand it also more.
I also investigated more the AcquireNextFrame. For better view I replaced the if statement with returnBlackBuffer. It looks like it's opposite. When it returns timeout, the picture is OK.
What is weird, if I have window with the "yellow" pattern and I move mouse over it, it's grabbed like the "red", but when I catch the window and move with the window, it's right grabbed like yellow.
Maybe it's some problem with Full screen grabbing and that the picture is in 8bit, but desktop maybe be in 10bit?
Anyway what is second investigation, that with that returnBlackBuffer I got white flashes e.g. when I'm in the visual studio.

The #373 also came to my mind, but making releaseFrame just before Acquire didn't change anything.

I will try tomorrow more :)

[Benik3]

Good vs bad captured image:

And it looks like the DXGI_ERROR_WAIT_TIMEOUT is not really connected with it (black is when error is returned)

BTW for some reason the saved pictures are wider then they should be (the black bar on the right). It happens even with DownscaleMipLevel = 0 (picture has 3456 instead of 3440 px) EDIT: I found that's because surfaceMap.Pitch return different number of bytes then is width*4 (e.g. it returns 1792=>448px, but texture is 430px (3440 >> 3))

[zomfg]

Good vs bad captured image:

what was the test setup here? old api? new? 8bit?..

And it looks like the DXGI_ERROR_WAIT_TIMEOUT is not really connected with it (black is when error is returned)

so it's both, crazy colors and timeouts (I don't have either in SDR) can you check the captured desc.Format for each frame, in case crazy colors spit out something different if they all are DXGI_FORMAT_B8G8R8A8_UNORM that's bad news (API bug?)

BTW for some reason the saved pictures are wider then they should be (the black bar on the right). It happens even with DownscaleMipLevel = 0 (picture has 3456 instead of 3440 px)

this is normal, it's just padding for memory alignment

[Benik3]

It was on the new API 8bit (in Prismatik) monitor is in 10bit.
For sure I tested some DDupl sample app and didn't notice this.

[Benik3]

On old API the result looks same:

When I use new API and R10G10B10A2 DXGI format, it always looks this ugly, so it looks like it doesn't convert the image to 8bit in some cases?

[zomfg]

that's what I'd suspect yeah, and if desc.Format says 8888 we have no way of knowing that it's borked I guess

have you tried this one? if that works consistently, try moving things in Prismatik til it works... or in the sample til it's broken in same way :D

[Benik3]

I just tried to writedesc.Format and texDesc.Format to txt file and is true, that this value changes.
Sometimes it returns 24 (10bit) and sometimes 87 (8bit). It looks like that that's the problem. When it returns 24, the image is broken.

screenData->duplication->GetDesc always return format 24.

[zomfg]

what if you set scaledTextureDesc.Format to 8888?

[Benik3]

If I set scaledTextureDesc.Format it still flickers (but is interesting that saved png of screen.imgData is OK), but when I forced also screen.imgFormat = mapDXGIFormatToBufferFormat(DXGI_FORMAT_B8G8R8A8_UNORM); it looks working OK :) Changing also texture.Format cause black image.

[zomfg]

so no issues now?.. you still get black frames?

[zomfg]

does this work?

// ...
texDesc.Format = DXGI_FORMAT_B8G8R8A8_UNORM;
// ...
scaledTextureDesc.Format = texDesc.Format;
// ...
screen.imgFormat = mapDXGIFormatToBufferFormat(texDesc.Format);
// ...

[Benik3]

I just tried to play with it (no HDR, but 10bit ON) and it looks good :) What works:

texDesc.Format = desc.Format;
scaledTextureDesc.Format = DXGI_FORMAT_B8G8R8A8_UNORM;
screen.imgFormat = mapDXGIFormatToBufferFormat(DXGI_FORMAT_B8G8R8A8_UNORM);

I will test more to see, if there are not any other issues.
I tested all combinations and this is the only one which works in 10bit.

EDIT: Hmm in some cases there is still black flickering :/
But in game I didn't noticed it.

[zomfg]

but does my snippet not work at all? I was hoping it would also cover the case without downscaling

black frames could be a combination of #373 and your grab interval (which is?) and refresh rate and/or maybe the limits of ddupl try adjusting ACQUIRE_TIMEOUT_INTERVAL and check if returnBlackBuffer() is called and from where

[Benik3]

Nope, it's doesn't work, resp. it shows black whenever there is 10bit image.
After digging into it I found, that CopyResource and CopySubresourceRegion can be used only on same format type group. So both should be R8 or R10, R16... etc. That's probably why it doesn't work when it tries copy DDupl data to texture, but for some reason it works when it makes CopySubresourceRegion to scaledTextureDesc. Maybe because Both types (R8 and R10) are 32bit per pixel. See Remarks here: https://docs.microsoft.com/en-us/windows/win32/api/d3d11/nf-d3d11-id3d11devicecontext-copyresource But it will not most likely works with R16G16B16A16 format (if someone will have it). So it's not 100% fix.

It looks like only proper fix is to convert the format (probably through pixel shader?), when source is not R8. But I think that's something I'm not able to program.

About grabbing I have now 10ms grab interval. I will try to test more.

EDIT: Maybe the "magic" is not in the CopySubresourceRegion but in the CreateShaderResourceView

[Benik3]

I tested more the flickering and I found, that the flickering is not black, but it's a previous frame...

[zomfg]

I tried these two

texDesc.Format = desc.Format;
scaledTextureDesc.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
screen.imgFormat = mapDXGIFormatToBufferFormat(DXGI_FORMAT_B8G8R8A8_UNORM);

texDesc.Format = desc.Format;
scaledTextureDesc.Format = DXGI_FORMAT_R10G10B10A2_UNORM;
screen.imgFormat = mapDXGIFormatToBufferFormat(DXGI_FORMAT_B8G8R8A8_UNORM);

so with a working conversion I should get funky colors if I save those as B8G8R8A8, right? but I get black frame every time

[Benik3]

hmm probably there is problem in copy from R8 to R10 texture? As I wrote is even weird that the copy texture copy from R10 to R8 works for me, because it shouldn't (from Microsoft description)...

[zomfg]

but even just BGR->RGB 8bit fails, so not even 8->10 while in your case both channel swap and depth change work apparently

[Benik3]

Ah, I see, weird

[Benik3]

Maybe this could help https://www.manongdao.com/q-1240529.html

[Benik3]

Maybe ID3D11DeviceContext::OMSetRenderTargets method could be helpful, from remark:

Each render target can have a different data format. These render target formats are not required to have identical bit-per-element counts.

[zomfg]

probably ms ddupl sample uses it

[Benik3]

OK, is there someone who knows these D3D staffs? I see it for the first time :D

[Benik3]

Just for info, I tried to turn on windows HDR and grabbing doesn't work even after this "hack" :/
I didn't investigate it more jet. But I was thinking, wouldn't be easier to user QImage:ConvertFormat on the final buffer? I will maybe try it.

Hmm it doesn't work. But when I save the picture as png they are OK (before and after conversion). But not if I save it in GrabberBase.cpp

                screen.imgDataSize = texDesc.Height * screenData->surfaceMap.Pitch;
            screen.scale = 1.0 / (1 << DownscaleMipLevel);
            screen.bytesPerRow = screenData->surfaceMap.Pitch;

            if (desc.Format == DXGI_FORMAT_R10G10B10A2_UNORM)
            {
                QImage imog(screenData->surfaceMap.pBits, screen.bytesPerRow / 4, screen.imgDataSize / screen.bytesPerRow, QImage::Format_A2BGR30_Premultiplied);
                imog.convertTo(QImage::Format_RGB32, Qt::AutoColor);
                screen.imgData = imog.bits();
                screen.imgFormat = mapDXGIFormatToBufferFormat(DXGI_FORMAT_R8G8B8A8_UNORM);

            }
            else
            {
                screen.imgData = screenData->surfaceMap.pBits;
                screen.imgFormat = mapDXGIFormatToBufferFormat(DXGI_FORMAT_B8G8R8A8_UNORM);
            }

            screenData->duplication->ReleaseFrame();

[zomfg]

does not work as in you get all black? or random colors? doing the conversion once the data is out of the texture would eat the cpu in a measurable way, especially when y'all get HDR 5k 240Hz and crank the grab interval 😄 so if it's avoidable...

[Benik3]

It's white, but I found that problem is in the pointer to img data (imog exist only in this function....).
Well If I got it working, I can measure how much CPU it will eat. Also I'm converting the downscaled picture and grabbing over let say 90FPS I don't see as "must have". You will anyway start to hit possibilities of COM port (on wifi maybe not) and LED strips. So better to have something which at least is able to grab correctly and slower then nothing :D

[Benik3]

Ok I got it working and with 12ms grabbing I have up to 15% CPU usage (i7-6700K 4,6GHz OC) :D EDIT: I forgot to turn off debug image saving to disk :) With img convert I have 1,7% CPU, without 0,8% so it's OK for me :)

[Benik3]

OMG I thought that I have it working after all, but the QIamge doesn't have format for R16G16B16A16 (RGB64 doesn't fit) :( Or maybe when I'm thinking now about it, problem is maybe in the HDR (so the values doesn't fit for SDR)

[zomfg]

you mean desc.Format is R16... now?

[Benik3]

yep. That's when you switch windows to HDR (and maybe also in other HDR applications, but I didn't test it yet)

[Shaggy-One]

Confirming the "pointer trails" fix at least for desktop. Haven't had time to test in games yet but with Windows HDR and Win8DDupl in Prismatik enabled the flashing went away with "pointer trails" on at minimum setting in internet browsing.

For those looking for this setting it's found in windows by typing "Mouse settings" into the start menu and clicking the system settings result. Scroll down to additional mouse options, click that and go to the "pointer options" tab in the window that pops up to find the "pointer trails" option.

Had the same issue as the title. All LEDs in the string flashing bright while mouse is moving with HDR and Win8 Desktop Duplication enabled. LEDs acting just fine in videos or games for the most part with the rare flash of "lightning". So far the pointer trails setting has fixed the issue. NO clue what that means but hopefully this helps those who know.

Current version of Prismatik (PSIEG build 5.11.2.27), Arduino w/FASTLED (not sure what version of FASTLED as I lost source code years ago) and 138 leds using APA102 LEDs at 77hz capture rate. Windows HDR enabled running at 3440x1440 144hz, 10 bit YCbCr444 color set via NVIDIA control panel.