Application Crash

[kdschlosser]

Operating system: wxPython version & source: 4.0.7.post1 msw (phoenix) wxWidgets 3.0.5 (pypi) Python version & source: 3.7.5 Stackless 3.7 (tags/v3.7.5-slp:f7925f2a02, Oct 20 2019, 15:28:53) [MSC v.1916 64 bit (AMD64)]

Description of the problem: I am getting an application crash I am not sure how I would recreate this application crash it appears to be an internal issue. Here is the information I do have on the application crash on the off chance it may help.

Problem signature:
  Problem Event Name:   APPCRASH
  Application Name: python.exe
  Application Version:  3.7.5150.1013
  Application Timestamp:    5dac6199
  Fault Module Name:    wxmsw30u_core_vc140_x64.dll
  Fault Module Version: 3.0.5.0
  Fault Module Timestamp:   5db79a16
  Exception Code:   c0000005
  Exception Offset: 00000000000394a0
  OS Version:   6.1.7601.2.1.0.256.1
  Locale ID:    1033
  Additional Information 1: 705e
  Additional Information 2: 705e6db06b5e83c9a4a9aa840488fd7b
  Additional Information 3: c57a
  Additional Information 4: c57aeafb8b973b3f380f9b5f6b3079f3

Here is the stack trace

wxmsw30u_core_vc140_x64.dll!wxGDIObject::IsOk() Line 49
    at c:\projects\bb2\dist-win64-py37\build\ext\wxwidgets\include\wx\gdiobj.h(49)
wxmsw30u_core_vc140_x64.dll!wxMSWDCImpl::SetPen(const wxPen & pen={...}) Line 1666
    at c:\projects\bb2\dist-win64-py37\build\ext\wxwidgets\src\msw\dc.cpp(1666)
[Inline Frame] wxmsw30u_core_vc140_x64.dll!wxMemoryDCImpl::Init() Line 72
    at c:\projects\bb2\dist-win64-py37\build\ext\wxwidgets\src\msw\dcmemory.cpp(72)
wxmsw30u_core_vc140_x64.dll!wxMemoryDCImpl::wxMemoryDCImpl(wxMemoryDC * owner) Line 46
    at c:\projects\bb2\dist-win64-py37\build\ext\wxwidgets\src\msw\dcmemory.cpp(46)
wxmsw30u_core_vc140_x64.dll!wxNativeDCFactory::CreateMemoryDC(wxMemoryDC * owner=0x00000000026ea7e0) Line 156
    at c:\projects\bb2\dist-win64-py37\build\ext\wxwidgets\src\common\dcbase.cpp(156)
wxmsw30u_core_vc140_x64.dll!wxMemoryDC::wxMemoryDC() Line 221
    at c:\projects\bb2\dist-win64-py37\build\ext\wxwidgets\src\common\dcbase.cpp(221)
_core.cp37-win_amd64.pyd!000007fed342316d()
siplib.cp37-win_amd64.pyd!000007fef7fe526f()
python37.dll!000007fedef5bb3a()
python37.dll!000007fedef5b858()
python37.dll!000007fedef44090()
python37.dll!000007fedef44c10()
python37.dll!000007fedef3d865()
python37.dll!000007fedef44e21()
python37.dll!000007fedef3d865()
python37.dll!000007fedef44e21()
python37.dll!000007fedef3d865()
python37.dll!000007fedef44e21()
python37.dll!000007fedef3d865()
python37.dll!000007fedef4620b()
python37.dll!000007fedef3d865()
python37.dll!000007fedef44e21()
python37.dll!000007fedef3d865()
python37.dll!000007fedef44e21()
python37.dll!000007fedef3d865()
python37.dll!000007fedef3d8d5()
python37.dll!000007fedef2f3e0()
python37.dll!000007fedef2efe9()
python37.dll!000007fedef2ef0c()
python37.dll!000007fedef2ee23()
python37.dll!000007fedef2eeb8()
python37.dll!000007fedef3dfd7()
python37.dll!000007fedef3d984()
python37.dll!000007fedef3d7a6()
python37.dll!000007fedef30677()
python37.dll!000007fedf02d84a()
python37.dll!000007fedf0ceaae()
ucrtbase.dll!thread_start<unsigned int (__cdecl*)(void * __ptr64)>()
kernel32.dll!BaseThreadInitThunk()
ntdll.dll!RtlUserThreadStart()

And here is the minidump without the heap.

debug.zip

If there is anything else I can provide let me know.

[Metallicow]

Do you know exactly what line it crashes on? Ex: by using print statements before/after...? If you can print("DEBUG") to the term/console window right before it crashes it might be helpful if you look at like +-4 lines before after as to what your trying to do... or possible the calling prev func/method. It appears that you are using a MemoryDC, so is a event involved for example like a paint event and if so, what event is it when you print it out..?

[RobinD42]

Also, it should be very uncommon these days with modern Windows, but is it possible your application is running out of handles?

[kdschlosser]

The crash is taking place when a MemoryDC gets created.The crash takes place on line 49 of gdiobj.h. I get an Unhandled Exception 0xC0000005: Access violation reading location 0xFFFFFFFFFFFFFFFF.

So when a MemoryDC instance gets created it calls wxMemoryDCImpl::Init() and in the Init method there is a call to set the pen to black SetPen(*wxBLACK_PEN); and in the SetPen method there is a check to see if the pen is ok pen.IsOk(). and this is what leads to the crash.

It's got nothing to do with handles.

The error happens at all kinds of different places it is not constant, nor is it constant on how long the app will run for before it happens.

[infinity77]

I think Robin is still correct in suggesting that you should look at how many GDI objects your application is creating. I’ve seen these kind of crashes happen in the past because I was creating way too many GDI objects. You can easily check that - and eventually exclude it as an issue - by monitoring the task manager on Windows: just go on the “Details” tab, right click on a header column and enable GDI objects.

[RobinD42]

Yep, pens, brushes, fonts, bitmaps, etc. all consume a GDI object handle. They should be released when the wx Python/C++ object is cleaned up, but if for some reason they are not being let go and you are creating several thousands of item, then it could lead to a failure when trying to create a new GDI object.

[kdschlosser]

The GDI Objects used changes between 564 and 570 currently I will wait once again until the application crashes again ans see if the task stays in the manager for me to see the number of handles

[kdschlosser]

OK so last night I started the program up. it has not crashed yet. Sometimes it will run for 5 minutes and crash.. others it will run for a few hours then crash, and sometimes it doesn't crash at all. When the crash happens it is always the same exact crash. I am not changing anything in the program at all and it is running using logged data and that data does not change between runs.

I am still sitting at the same number of GDI objects being used.. This time around running the program and it hasn't crashed it loaded the png files as wxBitmaps there are 523 image files but the ram consumption is at 343MB of RAM. While it is still high for the total of all files being 9.2MB it is not 1000MB of memory.

I am going to test something to see if I can poke the issue with a stick to instigate it to happen.

[Metallicow]

That sounds like it may be a memory leak if it takes a while to crash. Firefox for years had one and they finally got it fixed. But basically you could open just one tab and let it run for a day or two just sitting there and it would bluescreen your box. Needless to say alot of folks was happy when they finally got around to fixing it.

[kdschlosser]

Firefox and Chrome but still have massive memory leaks. I have to close my browser every few days to free up the 8GB of memory the browser consumes.

There is not a memory leak because the ram consumption of the app only varies 1-2mb while it is running. It doesn't creep up at all. The number of handles doesn't go up either. Everything stays right where it is. It just randomly crashes with an access violation error.

I wonder.. Is wxMemoryDC thread safe? I don't remember seeing anything that would make it thread safe in the areas of code surrounding where the exception is taking place. Now tell me if I am wrong here, wxBLACK_PEN is an already created handle yes? it is a high probability that I might be constructing 2 MemoryDC instances at the same time thus causing the use of wx.BLACK_PEN at the same time. would this cause the access violation? I would prefer to not have to use a thread lock before creating a MemoryDC instance because of the performance hit.

I just relocated a huge chunk of expensive code and set it up to render the images into a buffer at the start of the program. Then it never gets used again. I just finished up doing that so lets see if the problem goes away. This was a piece of code that was getting used a large number of times.

I am going to let my app loop for the next 2 days or so and see if it crashes.

[kdschlosser]

OK so it crashed again and when I debug it Now it says that it happened when constructing the memory DC and setting the brush and not the pen. This time I am getting "access violation executing location"

[kdschlosser]

It appears that adding the thread lock has fixed the issue. It is because of setting the pen or brush at the same i would imagine, This is because it is not creating a new wxPen or a new wxBrush instance and using a shared one.

[kdschlosser]

I am going to leave this open while the application runs for a day or 2 to verify the problem has in fact been solved.

[kdschlosser]

nope didn't fix the issue. It has got to be something else going on then. The problem is not in my code I am 100% sure about that. This is happening when I create a MemoryDC. I do not have an obscene number of handles open I have about 650. Where as Windows Explorer has some 4500.

[kdschlosser]

I forgot to mention I am not rendering in a paint event at all, or any other event. The rendering takes place when new data comes in to the program.

[kdschlosser]

give me a day to organize things with the program and I will zip it up including a sample of the data so it will run. Maybe someone can give it a go and let it run for a few hours and see if it crashes on their machine. If it was a coding error on my part then it would happen at the same spot every single time.

The fact that the error moves between setting the pen and setting the brush during the construction of the MemoryDC instance means there is something else going on. I am not setting a pen or a brush in the MemoryDC. I am setting the pens and the brushes I use in the GraphicsContext.

[RobinD42]

I wonder.. Is wxMemoryDC thread safe?

No, you should not create or use DCs or any other GDI or widget object from other than the main thread.

[kdschlosser]

@RobinD42

Why cannot this be done? It is not a limitation of Windows at all. GDI objects are not serialized and can be created/used in multiple thread designs and they can be shared between threads The application would need to make sure that if more then a single thread is using the same object then that object only be actually in use in a single thread at any given point in time.

What would be the cause of the limitation?

[kdschlosser]

Oh also it appears that Font Handles are not getting destroyed when they are out of scope.

[kdschlosser]

I think I know why... is there an HDC that is passed to CreateMemoryDC?? If there is this could be the cause. If NULL is passed instead of an HDC the calling thread then becomes the owner of the memory dc handle that is returned.

Nothing that I am doing is sharing any of the handles. It all is contained in the thread that is running it.

[kdschlosser]

nope that's not it. It is because wx.BLACK_PEN and wx.BLACK_BRUSH are created using the main thread and these are getting set in the internal code of wxWidgets. If this was not being done then it would be thread safe.

I am going to have to modify a copy of wxWidgets so it does not set the pen and the brush upon creation of the MemoryDC, this would eliminate the issue and make it thread safe.

The only trick when dealing with threads and GDI objects it would appear is that the thread that makes the object is the one that owns it and is the one that should be using it. wxWidgets was not coded in a way that allows this to work properly. I already know that asking for this to be changed is not going to happen so I am not going to even bother asking.

[kdschlosser]

I may have come up with a solution. It might be because of some kind of a memory fragmentation problem I am not sure why it is bombing. So what I am trying now is I only create a single memory DC instance for each thread that is running. I created a class that handles this delegation using __enter__ and __exit__ and threading.current_thread() I store the DCs in a dictionary with the thread instance as the key. This eliminates the need to create memory DC instances over and over again hopefully removing the problem.

The program started up without issue and seems to be running without any problems. now it is a waiting game until it crashes or doesn't crash.

[kdschlosser]

well it is still running. It uually would have crashed by now. I am going to let it run for the next 12 hours or so and see how things go with it.

[kdschlosser]

still running strong.. So I have that problem sorted I am pretty sure. gonna give it a while longer.

It appears as tho MemoryDC ends up being thread safe if you create a single instance of it and hold onto that instance while the thread is running. I would imagine that this would only work for the static creation of threads and not a dynamic one where threads are being created and destroyed all the time. I am not rendering to the screen using wxPython at all I am using the Windows API to do it, and this might have something to do with why it is working.

[RobinD42]

You may want to ask on wx-dev and see if they know of any other pitfalls you'll need to avoid or workaround.

[kdschlosser]

I will do that.

It has been running now for a few days and has not had an issue. It is actually a really complex rendering. I used Frames like layers in a paint program. It makes it easier to manage what needs to be updated and when.

any colors that are wrong and also the smoothness of it is not the program, it is the screen recorder to animated gif encoder that is causing that.

The program uses 21 threads, 742 GDI Object handles, 390mb of memory, and bounces between 0% and 2% for CPU use. It is using actual data I collected from the HSCAN data bus in my vehicle and that data is flowing into the program at a rate of about 300 or so updates a second.

[kdschlosser]

If you want to use the animation above for purposes of showing what is possible with wxPython you are more then welcome to do so.

This proves that a HMI (Human Machine Interface) in a vehicle does not have to be writting using a framework like QT and it also shows that Python is also up to the task.

[profprofgit]

@[kdschlosser] : I will be very much thankful to you if you can please explain, whether you did below described changes in your application or in wxwidget source code ? We are also facing same issue ( https://github.com/wxWidgets/wxWidgets/issues/22026 ), I made sure there doesn't exist any GDI leaks in my application. We are using 3.0.2 version of wxwidget code. In my application, only primary thread does the painting or dealing with GDI stuff( No other thread is working on DC related activity).

"I may have come up with a solution. It might be because of some kind of a memory fragmentation problem I am not sure why it is bombing. So what I am trying now is I only create a single memory DC instance for each thread that is running. I created a class that handles this delegation using enter and exit and threading.current_thread() I store the DCs in a dictionary with the thread instance as the key. This eliminates the need to create memory DC instances over and over again hopefully removing the problem.

The program started up without issue and seems to be running without any problems. now it is a waiting game until it crashes or doesn't crash."

[Metallicow]

From what you wrote, this may be on the hardware end. I don't know, I am on the road atm.

On Thu, Feb 10, 2022, 1:41 AM pranavT @.***> wrote:

@[kdschlosser] : I will be very much thankful to you if you can please explain, whether you did below described changes in your application or in wxwidget source code ? We are also facing same issue ( wxWidgets/wxWidgets#22026 https://github.com/wxWidgets/wxWidgets/issues/22026 ), I made sure there doesn't exist any GDI leaks in my application. We are using 3.0.2 version of wxwidget code.

"I may have come up with a solution. It might be because of some kind of a memory fragmentation problem I am not sure why it is bombing. So what I am trying now is I only create a single memory DC instance for each thread that is running. I created a class that handles this delegation using enter and exit and threading.current_thread() I store the DCs in a dictionary with the thread instance as the key. This eliminates the need to create memory DC instances over and over again hopefully removing the problem.

The program started up without issue and seems to be running without any problems. now it is a waiting game until it crashes or doesn't crash."

— Reply to this email directly, view it on GitHub https://github.com/wxWidgets/Phoenix/issues/1818#issuecomment-1034550893, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDTXRA7GZVJOL4M3MJFM3TU2NM2ZANCNFSM4SN22KNQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

[kdschlosser]

It's not a hardware problem. The creation, destruction and subsequent garbage collection of the DCs is what causes it. Something along the line is not getting done properly. It could either be Python, wxWidgets or Windows itself. I never got into debugging the problem to see what is exactly causing it. It could be that 2 calls are being made to create a DC at the exact same time.. I dunno. I know the crash happens at random intervals and I see no memory leak at all. If it was a video card issue on Windows the computer would BSOD. I am not sure if the problem exists with other operating systems.

@profprofgit The DC management system you have written sounds close to identical to what I had written... Right when a thread starts the DC gets created for that thread and because I have a thread that is dedicated to each frame everything works as it should. I don't create any kind of a PaintDC in this manner only MemoryDCs. I am using the MemoryDCs to write the updated information to a bitmap and then I am using a ClientDC to draw the bitmap.

So there is a loop that each thread runs in and using threading.Event as the loop exit flag and using Threading.Lock to stall the thread so no spinning wheels occurs. Right when the thread starts I have it acquire the lock before entering the loop. Once in the loop the code to do the drawing is run and the lock is acquired again before the loop restarts. If new information comes in that has to be displayed that information get put into whatever container and the lock gets released which releases the thread in the loop so it can do it's thing.