Alternative to using CreateBitmapSourceFromHBitmap for rendering in WPF

[ftlPhysicsGuy]

Topic: With the help of a friend, I've found an alternative to using CreateBitmapSourceFromHBitmap when implementing SharpGL in a WPF application!

Working through a couple of problems/solutions has lead me to use an alternative method for creating a bitmap source from a RenderContextProvider (DIB Section or FBO) without always calling System.Windows.Interop.Imaging.CreateBitmapSourceFromHBitmap(...) (via a call to HBitmapToBitmapSource(IntPtr hBitmap) in BitmapConversion.cs):

Issue 1: the GC.Collect() call made with every call to HBitmapToBitmapSource slows down rendering and can be significant when memory usage increases.

Solution 1: I found that by removing the GC.Collect call, I could add a catch for any OutOfMemoryException and perform a GC.Collect() (and set bitSrc = null) at that point. However, I then had to be careful to check for null bitSrc returns and I never actually hit a memory issue to cause the problem -- possibly because CreateBitmapSourceFromHBitmap was doing garbage collection internally (see Issue 2 below).

Issue 2: When memory usage got fairly high (around 1.2 GB used - which could happen with my particular application), EVERY CALL to CreateBitmapSourceFromHBitmap would cause a garbage collection, often taking 900+ ms to complete. I validated this by looking at GC.CollectionCount(0) before and after the call to CreateBitmapSourceFromHBitmap and using a System.Diagnosis.Stopwatch to verify that the processing time was being taken up by that specific call.

Solution 2: Here, I had to get away from the call to CreateBitmapSourceFromHBitmap. A friend of mine found an alternative and provided an initial implementation (BIG thanks to him!). After playing around with it, I've come up with my current method for rendering to the image in OpenGLControl.xaml.cs. Essentially, I use System.Windows.Media.Imaging.WriteableBitmap as the image.Source and refill it (vice re-creating it) on every tick. I call System.Runtime.InteropServices.Marshal.Copy(...) to copy from a bits pointer to an image buffer and use that to write to the WriteableBitmap. This then becomes the source of the image. You have to set up the buffer and WriteableBitmap when the size of the display changes, and you have to use image.RenderTransform and image.RenderTransformOrigin to flip the image vertically for an FBORenderContextProvider, but the whole process seems as fast (if not faster) than the current method.

I still have to use ONE call too HBitmapToBitmapSource (if we believe the dpiX, dpiY, and/or PixelFormat of the bitmap will ever be different from 96, 96 and PixelFormats.Bgra32 respectively) but it's only once and it causes no issues.

Here's my entire solution to modifying the OpenGLControl.xaml.cs file:

• Add the following fields: // Fields to support the WritableBitmap method of rendering the image for display byte[] m_imageBuffer; WriteableBitmap m_writeableBitmap; Int32Rect m_imageRect; int m_imageStride; double m_dpiX = 96; double m_dpiY = 96; PixelFormat m_format = PixelFormats.Bgra32; int m_bytesPerPixel = 32 >> 3;
• In the UpdateOpenGLControl(...) method, just before the lock(gl) is closed, add the following: // Force re-creation of image buffer since size has changed m_imageBuffer = null;
• In the OnApplyTemplate() method, add the following lines just after the gl.Create(...) call: // Force re-set of dpi and format settings m_dpiX = 0;

• Add the following method: ///

  /// FOR NEW RENDER METHOD /// Fill the ImageSource from the provided bits IntPtr, using the provided hBitMap IntPtr /// if needed to determine key data from the bitmap source. ///

  /// An IntPtr to the bits for the bitmap image. Generally provided from /// DIBSectionRenderContextProvider.DIBSection.Bits or from /// FBORenderContextProvider.InternalDIBSection.Bits. /// An IntPtr to the HBitmap for the image. Generally provided from /// DIBSectionRenderContextProvider.DIBSection.HBitmap or from /// FBORenderContextProvider.InternalDIBSection.HBitmap. public void FillImageSource(IntPtr bits, IntPtr hBitmap) { // If DPI hasn't been set, use a call to HBitmapToBitmapSource to fill the info // This should happen only ONCE (near the start of the application) if (m_dpiX == 0) { var bitmapSource = BitmapConversion.HBitmapToBitmapSource(hBitmap); m_dpiX = bitmapSource.DpiX; m_dpiY = bitmapSource.DpiY; m_format = bitmapSource.Format; m_bytesPerPixel = gl.RenderContextProvider.BitDepth >> 3; // FBO render context flips the image vertically, so transform to compensate if (RenderContextType == SharpGL.RenderContextType.FBO) { image.RenderTransform = new ScaleTransform(1.0, -1.0); image.RenderTransformOrigin = new Point(0.0, 0.5); } else { image.RenderTransform = Transform.Identity; image.RenderTransformOrigin = new Point(0.0, 0.0); } }

    // If the image buffer is null, create it
    // This should happen when the size of the image changes
    if (m_imageBuffer == null)
    {
        int width = gl.RenderContextProvider.Width;
        int height = gl.RenderContextProvider.Height;
  
        int imageBufferSize = width * height * m_bytesPerPixel;
        m_imageBuffer = new byte[imageBufferSize];
        m_writeableBitmap = new WriteableBitmap(width, height, m_dpiX, m_dpiY, m_format, null);
        m_imageRect = new Int32Rect(0, 0, width, height);
        m_imageStride = width * m_bytesPerPixel;
    }
  
    // Fill the image buffer from the bits and create the writeable bitmap
    System.Runtime.InteropServices.Marshal.Copy(bits, m_imageBuffer, 0, m_imageBuffer.Length);
    m_writeableBitmap.WritePixels(m_imageRect, m_imageBuffer, m_imageStride, 0);
  
    image.Source = m_writeableBitmap;
  }

• In the timer_Tick(...) method, replace the case statement for RenderContextType.DIBSection with the following: case RenderContextType.DIBSection: { var provider = gl.RenderContextProvider as DIBSectionRenderContextProvider; var hBitmap = provider.DIBSection.HBitmap;

                    if (hBitmap != IntPtr.Zero)
                    {
                        // FOR NEW RENDER METHOD:
                        FillImageSource(provider.DIBSection.Bits, hBitmap);
  
                        //var newFormatedBitmapSource = GetFormatedBitmapSource(hBitmap);
  
                        ////  Copy the pixels over.
                        //image.Source = newFormatedBitmapSource;
                    }
                }
                break;

• In the timer_Tick(...) method, replace the case statement for RenderContextType.FBO with the following: case RenderContextType.FBO: { var provider = gl.RenderContextProvider as FBORenderContextProvider; var hBitmap = provider.InternalDIBSection.HBitmap;

                    if (hBitmap != IntPtr.Zero)
                    {
                        // FOR NEW RENDER METHOD:
                        // The FBORenderContextProvider flips the image vertically, so transform it
                        FillImageSource(provider.InternalDIBSection.Bits, hBitmap);
  
                        //var newFormatedBitmapSource = GetFormatedBitmapSource(hBitmap);
  
                        ////  Copy the pixels over.
                        //image.Source = newFormatedBitmapSource;
                    }
                }
                break;

I also have a test application that creates a spinning line (like a radar screen) and provides a "Run Test" button (which creates 16,000,000 objects to seriously increase memory usage). If you use the old OpenGLControl, the spinner slows to about one tick per second after running the test. With the new OpenGLControl I'm using, there's no noticeable change after the test (unless you resize -- then there's a slight delay before it gets back to normal since it has to re-create the WriteableBitmap).

Finally, note that with this modification, there is no longer a need to modify the BitmapConversion code.

If it would be helpful, I could post my modified OpenGLControl.xaml.cs file.

Thanks.

-Jason

[Thormidable]

Hi Jason, If you could post your modified OpenGLControl.xaml.cs file, that would be great. We are currently working on reducing the SharpGL.WPF overheads in our application and were planning on using WriteableBitmap to reduce the work done. It would save us a fair bit of work if you would be good enough to upload your version of this class.

Thank you

[ftlPhysicsGuy]

Hi, Thormidable....

I'm attaching the modified file here. Hope it helps!

-Jason

OpenGLControl.xaml.cs.zip

[Thormidable]

Hi Jason, Thank you, I've seen a substantial improvement in frame rate already

[xueqingsun]

@Thormidable Was this code change adopted by SharpGL and included in a new release? I tried to remove GC.Collect() in BitmapConversion.cs, it worked, the frame rate was improved greatly.

[Thormidable]

@xueqingsun it is in @ewngs repo. Grab that, it's got all the updates from this repo, plus a few more fixes. https://github.com/ewngs/sharpgl/commits/master

[xueqingsun]

Many thanks to @Thormidable and @ewngs, The performance improved greatly. On my machine, the FPS is 20 and previously it was 8 to 10. ewngs's code change worked.

Instead the change of ewngs, merely remove GC.Collect() will work as well as the ewngs's change. Normally we don't call GC to collect. So why is GC.Collect() there?

[xueqingsun]

By the way, the way of using stopwatch in OpenGLControl won't give a correct FPS. It will give a much higher FPS than the real FPS.

[ftlPhysicsGuy]

Just to note: in my situation, the app would get into a state where the call to

CreateBitmapSourceFromHBitmap was internally causing its own garbage collection every time it was called. I could remove the manual GC, but the issue persisted.

-Jason

On Wed, Jan 18, 2017 at 7:27 PM Xueqing Sun notifications@github.com wrote:

Many thanks to @Thormidable https://github.com/Thormidable and @ewngs https://github.com/ewngs, The performance improved greatly. On my machine, the FPS is 20 and previously it was 8 to 10. ewngs's code change worked.

Instead the change of ewngs, merely remove GC.Collect() will work as well as the ewngs's change. Normally we don't call GC to collect. So why is GC.Collect() there?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/dwmkerr/sharpgl/issues/121#issuecomment-273655482, or mute the thread https://github.com/notifications/unsubscribe-auth/AHeoTN1riXSBiZb5rZ3XxBV9Gs-_ssgoks5rTrv-gaJpZM4H8Zsn .