Problem with fast draws.

[Cupiii]

Hi Christian,

i have a problem related to your project, but surely not caused by your project.

I try to compose pre loaded Images with different transformations (i.e. rotations, scales, translations) into one Bitmap using a Graphics Class inside a Timer-thread.

Problem is, that i can't access the images in a thread, but have to invoke the UI-Thread via a delegate to do so. But the UI-Thread is only checked every 500 ms, which is way too slow for me.

        public void mydraw()
        {
            if (Thread.CurrentThread != myUIthred) //Tell the UI thread to invoke mydraw if its not himself.
            {
                BeginInvoke(myaction);
                 return;
            }
            tempg.Clear(Color.Black);        //         tempg is created elsewhere:    tempg = Graphics.FromImage(result);
            tempg.TranslateTransform(1024, 1024);
            tempg.RotateTransform((float)angle);
            tempg.TranslateTransform(-1024, -1024);
            tempg.DrawImage(back, 0, 0);    // back, middle, ring and over are preloaded images
            tempg.DrawImage(middle, 0, 0);
            tempg.DrawImage(ring, 0, 0);
            //tempg.Flush();
            tempg.TranslateTransform(1024, 1024);
            tempg.RotateTransform((float)-angle);
            tempg.TranslateTransform(-1024, -1024);
            tempg.DrawImage(over, 0, 0);
            //tempg.Flush();
            tempg.Save();

            deck.SetKeyBitmap(9, KeyBitmap.Create.FromBitmap((Bitmap)result));

            angle = angle +2;  // for testing - angles are read from simulator in a another thread
       }

I guess, while showing the videoframes on the streamdeck, you encountered the same problem.

Do you have a reciepe or tip for me, how i can achive drawing in a higher frame rate while using Bitmaps? Maybe you could push me into the right direction.

I hope it is ok for you that i ask this question via a git issue.

Gern auch auf deutsch :-)

Viele Grüße aus Bremen

Andreas

[c0nnex]

Ran into the sam Problem. easy solution.

The KeyBitmap.FromBitmap must run in the same thread where the Bitmap was instatiated. I move all bitmap operations to a seprate thread and precache the Keybitmaps there, pushing them then just out to the device when needed. The keybitmap itself is threadsafe. Works a treat.

Only still have the "half" image issue, that sometimes images will look halftoned for the upper or lower half of the bitmap, while the bitmap itself is 100% ok, and the keybitmap is ok as well.

[Cupiii]

Thank you this helped a lot!!!

I also added a poor-mans lock, if the thread is called again, while not yet ready, since the Graphics Object is not thread save.

I guess i should also not use 2048x2048 pixel images while trying to rotate them 30 times a second. :-)

        public void mydraw()
        {
            if (imagesnotloaded)
            {
                myimg = Bitmap.FromFile("c:\\users\\andre\\fssd\\att.png");
                back = Bitmap.FromFile("c:\\users\\andre\\fssd\\attback2.png");
                middle = Bitmap.FromFile("c:\\users\\andre\\fssd\\attmiddle2.png");
                ring = Bitmap.FromFile("c:\\users\\andre\\fssd\\attring2.png");
                over = Bitmap.FromFile("c:\\users\\andre\\fssd\\attover2.png");

                result = new Bitmap(144, 144);
                result2 = new Bitmap(144, 144);

                tempKB2 = KeyBitmap.Create.FromBitmap((Bitmap)result);
                tile = new Bitmap(back.Width / 4, back.Height / 4);
                imagesnotloaded = false;
            }
            if (!lockme)
            {
                lockme = true;
                Bitmap result3 = new Bitmap(144, 144);
                var tempg = Graphics.FromImage(result3);
                tempg.Clear(Color.Black);
                tempg.TranslateTransform(72, 72);
                tempg.RotateTransform((float)angle);
                tempg.TranslateTransform(-72, -72);
                tempg.DrawImage(back, 0, 0);
                tempg.DrawImage(middle, 0, 0);
                tempg.DrawImage(ring, 0, 0);
                //tempg.Flush();
                tempg.TranslateTransform(72, 72);
                tempg.RotateTransform((float)-angle);
                tempg.TranslateTransform(-72, -72);
                tempg.DrawImage(over, 0, 0);
                //tempg.Flush();
                tempg.Save();

                deck.SetKeyBitmap(9, KeyBitmap.Create.FromBitmap((Bitmap)result3));
                tempg.Dispose();

                angle = angle + 3;
                lockme = false;
            }
        }

Thank you so much. That was excactly a push into the right direction.

I have no problems with half-Images. Even with 30hz refresh rate.

[c0nnex]

If you dont mind some hints:

lockme is not thread safe. use

private object lockObject = new object()

public void draw()
{
  lock(lockObject)
  {
     .......
     using (var tempg = Graphics.FromImage(result3)) 
     {
      }
     result3.Dispose(); // !!!!
  }
}

Will prevent reenrant code and forgetting about the Disposal. Also VITAL: DISPOSE the bitmaps when you do not need them anymore! Garbage collection is too slow and might leak handles, and then you will run out of GDI handles sooner or later

[Cupiii]

I dont mind :-). Thank you. Proper locking was on my todo list.

But may i aks you: If i understand correctly: If lock(lockobject) is called again while the lockobject is locked, it will wait. What if draw() is called faster than the code inside the lock (rotating and drawing graphics is quite slow)? Won't i get a stack overflow sooner or later ?

I think, I could also get rid of result3 when using KeyBitmap.Create.FromGraphcis. Will test tomorrow.

Thank you so much. Dank' Dir!

[Cupiii]

Thinking about this: since i call draw() with a timer i could also set AutoReset to false. This should give me no locking problems anymore. Timing is not so important for my draw() method.

[c0nnex]

You wil not get a stackoverflow , but a big queue in front of the lock. Do not use timer for such things. Use a Background-Thread , and have it sleep the timer-value in each cycle.

[wischi-chr]

Only still have the "half" image issue, that sometimes images will look halftoned for the upper or lower half of the bitmap, while the bitmap itself is 100% ok, and the keybitmap is ok as well.

@c0nnex this shouldn't happen. What stream deck hardware do you use? Could you provide code that would allow me to reproduce the issue?

[wischi-chr]

@Cupiii could you provide a link to the repository it's pretty hard to help only knowing small parts of what you are doing and what you want to achieve.

I'm not sure why you would need a locking mechanism in "mydraw". Drawing shouldn't happen in parallel, a background timer with 10-40 FPS should do the trick. There are a lot more things I noticed in the short section you provided, like why do you call tempg.Save(); without a state restore this very likely doesn't do what you think it does.

Initialization should probably happen in another method an the path shouldn't be hardcoded. Even during early development it should at least be a constant because it will be way easier to change later.

Also according to your comment the images are 2048x2048, and every drawing tick you scale transform and draw them. You probably should cache the downscaled images and only rotate them. You could go even further and trade of memory for speed. Because you always add 3 to your angle there are only 360/3 = 120 states which you could all cache if you want.

But as I said earlier. If you provide a link to your repo I may find the time help you clean things up and get it running.

[c0nnex]

Only still have the "half" image issue, that sometimes images will look halftoned for the upper or lower half of the bitmap, while the bitmap itself is 100% ok, and the keybitmap is ok as well.

@c0nnex this shouldn't happen. What stream deck hardware do you use? Could you provide code that would allow me to reproduce the issue?

I have it on my Streamdeck Live (very first HW rev), but some of my users observe it on XL as well sometimes. It only happens if all bitmaps for all keys are updated, and the half-toning is always the upper or lower half. Never the whole button. Oddly it seems to be always the same buttons that shows that issue, which makes me think it's some kind of native memory-problem in the overlapped write. Didn't check with usb-tracer yet what gets sent to the device, since it's not reliably reproducible and only occours occasionally.

Basically the code does nothing else than Foreach button set pregenerated keybitmap. (same bitmap is shown in UI of the app and it's always 100% ok)

I'll see if I can seperate the code without putting too much time into it.

[Cupiii]

@Cupiii could you provide a link to the repository it's pretty hard to help only knowing small parts of what you are doing and what you want to achieve.

Thanks so much Wischi, and also C0nnex.

In fact the code snippet was only a proof of concept. So there is no repository (yet). I also have to say that my programming skills are somewhat old... more Pascal and Delphi instead of C#. Don't know much about threading and such.

But thank's so much. The 3 degrees increment is just for testing, since Microsoft Flight Simulator 2020 is still buggy. And there is some more calculation on the images: All 360 degrees are needed and the middle images will be translated at about +- 20 pixels (which was not yet in the code i submitted). - Resulting in 360 * 41 Pictures.

Here is a photo (i guess c0nnex will recognise what it is in less than a second :-) ). In fact I realised connex is doing "somehow" :-) the same, while looking at his website. Maybe he will think about also displaying 3x3 live-gauges on the stream deck with his software... somehow selectable. wink, wink. Does not look too shabby for people with limited resources for other flight sim Hardware.

[c0nnex]

Only still have the "half" image issue, that sometimes images will look halftoned for the upper or lower half of the bitmap, while the bitmap itself is 100% ok, and the keybitmap is ok as well.

@c0nnex this shouldn't happen. What stream deck hardware do you use? Could you provide code that would allow me to reproduce the issue?

Here an image of a user. keymaps are completely out of order, while the same bitmap being used to show in ui is ok. MIght be some problem in scaling code? (User uses 288x288px pngs) http://prntscr.com/u96h27

[c0nnex]

Here is a photo (i guess c0nnex will recognise what it is in less than a second :-) ). In fact I realised connex is doing "somehow" :-) the same, while looking at his website. Maybe he will think about also displaying 3x3 live-gauges on the stream deck with his software... somehow selectable. wink, wink. Does not look too shabby for people with limited resources for other flight sim Hardware.

Not really. My users already invested a lot of money in expensive picture frames (aeh FIP), although technically rendering a gauge on the deck would be easy going. Nice idea though. I'll hear what they think about it.

[c0nnex]

Additional info, if that Image disortion appears (the shading), if I enforce a resend off all keybitmaps a second time, they will be ok.

[c0nnex]

While tracking down tha shading/corruption problem, I suspected the Split or Write having a problem. For testing I replaced OutputReportSplitter.SPlit with a non yield, non buffer reuse version:

public static IEnumerable<byte[]> Split(
            byte[] data,
            byte[] buffer,
            int bufferLength,
            int headerSize,
            int keyId,
            PrepareDataForTransmittion prepareData)
        {
            var maxPayloadLength = bufferLength - headerSize;

            var remainingBytes = data.Length;
            var bytesSent = 0;
            var splitNumber = 0;
            List<byte[]> bb = new List<byte[]>();

            while (remainingBytes > 0)
            {
                var isLast = remainingBytes <= maxPayloadLength;
                var bytesToSend = Math.Min(remainingBytes, maxPayloadLength);
                var b = new byte[buffer.Length];
                Array.Copy(data, bytesSent, b, headerSize, bytesToSend);
                prepareData(b, splitNumber, bytesToSend, keyId, isLast);
                bb.Add(b);

                bytesSent += bytesToSend;
                remainingBytes -= bytesToSend;
                splitNumber++;
            }
            return bb;
        }

Voila, problem went away. I think the Overlapped Write will return while it is not really done (E_INPROGRESS?), then reusing the buffer might corrupt the buffer still being sent. I had that with other devices where i have to send loads of data (250k+) and had a really hard time to get the overlapped io reliably working from C#

[c0nnex]

Quick update about the corruption problem. We did a lot of tests and found out the following:

• If SD is on a HUB , image corruption is occouring frequently
• if SD is on a HUB and Elgato Software is running, image corruption is occurring a lot more often (almost every 2nd push)
• if SD is connected directly to the compuer image corruption is occurring, but a lot less

For next test I will add a 25ms delay after sending each image. It seems to be something with timing and buffer/flow control on the device, as i can confirm that data sent to the device is never corrupted ( checked with USB-Logging device put in between)

[wischi-chr]

@c0nnex do you have a repo that can reproduce the behaviour? It's quite a long time since I've seen image corruption glitches. In fact since the commit 11716d3471b66b85df26fc341f1a8536b810e00f I haven't seen a single case.