libgdiplus cannot allocate (small) bitmap

holgerschurig commented 4 years ago

Steps to reproduce

csc Program.cs Form1.cs Form1.Designer.cs
mono Program.exe
press the button

The 3 files can be found at https://gist.github.com/holgerschurig/d7358b623f874eb447d66511d9650b25

Current Behavior

schurig@desktop:~/gdiplusissue$ mono Program.exe 
Create bitmap: width=952 height=1051
System.OutOfMemoryException: Not enough memory to complete operation [GDI+ status: OutOfMemory]
  at System.Drawing.GDIPlus.CheckStatus (System.Drawing.Status status) [0x000b7] in <62b978d6013a443d8a019e6b99378156>:0 
  at System.Drawing.Graphics.FromImage (System.Drawing.Image image) [0x00039] in <62b978d6013a443d8a019e6b99378156>:0 
  at GraphicsTestApp.Form1.CreateOffScreenBitmap () [0x00039] in <06e8a7543539404ab6a28e14fe14144a>:0 
  at GraphicsTestApp.Form1.button2_Click (System.Object sender, System.EventArgs e) [0x00001] in <06e8a7543539404ab6a28e14fe14144a>:0 
  at System.Windows.Forms.Control.OnClick (System.EventArgs e) [0x00019] in <29ae5f13c05642a1be782372651d31a2>:0 
  at System.Windows.Forms.Button.OnClick (System.EventArgs e) [0x0001e] in <29ae5f13c05642a1be782372651d31a2>:0 
  at System.Windows.Forms.ButtonBase.OnMouseUp (System.Windows.Forms.MouseEventArgs mevent) [0x00069] in <29ae5f13c05642a1be782372651d31a2>:0 
  at System.Windows.Forms.Button.OnMouseUp (System.Windows.Forms.MouseEventArgs mevent) [0x00000] in <29ae5f13c05642a1be782372651d31a2>:0 
  at System.Windows.Forms.Control.WmLButtonUp (System.Windows.Forms.Message& m) [0x00078] in <29ae5f13c05642a1be782372651d31a2>:0 
  at System.Windows.Forms.Control.WndProc (System.Windows.Forms.Message& m) [0x001b4] in <29ae5f13c05642a1be782372651d31a2>:0 
  at System.Windows.Forms.ButtonBase.WndProc (System.Windows.Forms.Message& m) [0x00037] in <29ae5f13c05642a1be782372651d31a2>:0 
  at System.Windows.Forms.Button.WndProc (System.Windows.Forms.Message& m) [0x00000] in <29ae5f13c05642a1be782372651d31a2>:0 
  at System.Windows.Forms.Control+ControlWindowTarget.OnMessage (System.Windows.Forms.Message& m) [0x00000] in <29ae5f13c05642a1be782372651d31a2>:0 
  at System.Windows.Forms.Control+ControlNativeWindow.WndProc (System.Windows.Forms.Message& m) [0x0000b] in <29ae5f13c05642a1be782372651d31a2>:0 
  at System.Windows.Forms.NativeWindow.WndProc (System.IntPtr hWnd, System.Windows.Forms.Msg msg, System.IntPtr wParam, System.IntPtr lParam) [0x00085] in <29ae5f13c05642a1be782372651d31a2>:0

Expected Behavior

I expect Mono to be able to allocate a bitmap of just 952x1051, especially when I have more than 6 GB of free RAM.

Additional Info

The above stacktrace is from my normal Debian 10 (Buster) x86 development box. But it happened first on a i.MX6Q (armhf architecture) box. On the same device, there used to be no problem. I found out that copying an old libgdiplus.so.0.0.0 from an old .deb (I think it was 4.2) "fixes" the problem.

I think the first version where I noticed this error was 6.0.4

On which platforms did you notice this

[ ] macOS [x] Linux [ ] Windows

Version Used

schurig@desktop:~/gdiplusissue$ mono --version
Mono JIT compiler version 6.10.0.104 (tarball Fri Jun 26 19:40:04 UTC 2020)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
    TLS:           __thread
    SIGSEGV:       altstack
    Notifications: epoll
    Architecture:  x86
    Disabled:      none
    Misc:          softdebug 
    Interpreter:   yes
    LLVM:          yes(610)
    Suspend:       hybrid
    GC:            sgen (concurrent by default)
schurig@desktop:~/gdiplusissue$ dpkg -l libgdiplus | cat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version                    Architecture Description
+++-==============-==========================-============-============================================
ii  libgdiplus     6.0.5-0xamarin1+debian10b1 i386         interface library for System.Drawing of Mono

holgerschurig commented 4 years ago

The commit that introduced this behavior was https://github.com/mono/libgdiplus/commit/215632d4f8a98d4083e91395f4efe33fe369a091

Doing this:

git checkout 215632d4f8a98d4083e91395f4efe33fe369a091^
git clean -fdx
./autogen.sh
make -j8 && sudo cp -v src/.libs/libgdiplus.so.0.0.0 /usr/lib

will make the test program from above (gist https://gist.github.com/holgerschurig/d7358b623f874eb447d66511d9650b25) work without a crash. Note the ^ at the checkout to get the previous version.

holgerschurig commented 4 years ago

Manually reverting hunks revealed that the following change introduced the crash:

@@ -800,27 +799,35 @@ GdipCreateBitmapFromScan0 (int width, int height, int stride, PixelFormat format
                case PixelFormat16bppRGB565:
                        /* fake them as 32bpp RGB as Cairo deprecated CAIRO_FORMAT_RGB16_565 support */
                        /* why 32bpp ? because that's the result of MS GDI+ when loading them, even if the bitmap is
-                       format = PixelFormat32bppRGB;
-                       stride *= 2;
                        cairo_format = CAIRO_FORMAT_ARGB32;
                        break;

After I re-added these two lines, things worked again.

This worked even when I re-added these two lines to git master.

46cv8 commented 1 year ago

Thanks for the tip! Reverting this change didn't work for me, but changing "PixelFormat.Format16bppRgb555" in the host application to "PixelFormat.Format32bppRgb" worked around the issue for me. :)

mono / libgdiplus