Qt5 application crashes when using MESA

[BlohoJo]

Hi!

I was hoping to be able to run a program called Topaz Labs JPEG to RAW AI on a VMware server using software/CPU mode. It's a Qt5 application. The VMware server is running Windows Server 2008 R2.

I downloaded and installed the trial. When I try to run the program, it crashes instantly with this:

Faulting application name: Topaz JPEG to RAW AI.exe, version: 0.0.0.0, time stamp: 0x5c4f4ade Faulting module name: unknown, version: 0.0.0.0, time stamp: 0x00000000 Exception code: 0xc0000005 Fault offset: 0x0000000000000000 Faulting process id: 0x16c0 Faulting application start time: 0x01d5e2c1e27805ff Faulting application path: C:\Program Files\Topaz Labs\Topaz JPEG to RAW AI\Topaz JPEG to RAW AI.exe Faulting module path: unknown Report Id: 2167a7cf-4eb5-11ea-a2a8-0050560b8935

I can see in the installation directory there is an opengl32sw.dll file. Not sure if this is MESA based or what, but I tried renaming it to opengl32.dll. The program looks like it gets a little farther but it still crashes, this time with the following:

Faulting application name: Topaz JPEG to RAW AI.exe, version: 0.0.0.0, time stamp: 0x5c4f4ade Faulting module name: Qt5Gui.dll, version: 5.12.0.0, time stamp: 0x5c0514b1 Exception code: 0xc0000005 Fault offset: 0x00000000002f1310 Faulting process id: 0x938 Faulting application start time: 0x01d5e2c2041f97ef Faulting application path: C:\Program Files\Topaz Labs\Topaz JPEG to RAW AI\Topaz JPEG to RAW AI.exe Faulting module path: C:\Program Files\Topaz Labs\Topaz JPEG to RAW AI\Qt5Gui.dll Report Id: 42dcdf1f-4eb5-11ea-a2a8-0050560b8935

Finally, I tried copying in opengl32.dll from the latest 64 bit MESA mingw release into the application directory. This also didn't work; I get the same crash as above (0xc0000005 in Qt5Gui.dll).

(I also tried the MSVC release; it didn't have a different result.)

The program appears to have a Windows registry key for software mode:

[HKEY_CURRENT_USER\Software\Topaz Labs\Topaz JPEG to RAW AI\appMain] "j2r_gpuOrCpu"=dword:00000001

(I assume 1 is for CPU; it's set to 0 by default.)

That also didn't have any effect.

Again, I did verify with Dependency Walker that the MESA opengl32.dll was loading from the application directory. It was.

From what I've researched online, both JPEG to RAW as well as Qt5 should support software mode. I have so far had a lot of success using software that uses OpenGL on a VMware server by using MESA.

Have you heard of anything like this before and know of a way to fix this?

My virtual machine is running Windows Server 2008 R2 (which is Windows 7 x64 based).

[BlohoJo]

I figured out the solution to the problem myself.

MESA has environment variables that determine how it works for different OpenGL versions.

Because JPEG to RAW uses OpenGL 3.3, the following MESA environment variable needs to be set before running it:

MESA_GL_VERSION_OVERRIDE=3.3COMPAT

So, I created a batch file to start JPEG to RAW:

@set MESA_GL_VERSION_OVERRIDE=3.3COMPAT @start "" "Topaz JPEG to RAW AI.exe" @exit

The program now works in software mode!

I SERIOUSLY didn't think there was a solution to this. I had noticed that the program specified that it needs OpenGL 3.3 and I searched to see if MESA supported this, and that's where I learned of the environment variables. I had never needed to set any before.

So there it is for anyone in the future who might run into this issue.

[pal1000]

If you would have read the docs from the beginning it would have saved from a lot of hassle:

• https://github.com/pal1000/mesa-dist-win#opengl-context-configuration-override
• https://www.mesa3d.org/envvars.html

[BlohoJo]

I doubt it, as I had not paid attention to what version of OpenGL the app needed.

I just spent hours trying to get another one of their apps to work - Sharpen AI. I thought this would be a breeze since I had gotten it to work with JPEG to RAW. But this one didn't work. Even with environment variables set, I got the Qt5Gui.dll crash. After about 12 hours of coming back to it and trying different things and just about giving up figuring this was an app that just wouldn't work with MESA, I discovered that I had to use the MSVC build of MESA, copy libglapi.dll, libGLESv2.dll, and opengl32.dll to the application directory, and finally, RENAME opengl32.dll to opengl32sw.dll, replacing the existing opengl32sw.dll in the application directory, and finally set the MESA_GL_VERSION_OVERRIDE=3.3COMPAT environment variable.

In JPEG to RAW, opengl32sw.dll doesn't seem to be used, and simply using the mingw build of opengl32.dll plus setting the environment variable was enough. With Sharpen AI, I can use Dependency Walker and observe that it reports that it wants to load opengl32.dll from the application directory. But when running the program it acts like it isn't even there and crashes the same as it does when opengl32.dll isn't present. But if I use the MSVC build and rename the file to opengl32sw.dll, it works. I can even load the application into x64dbg and search all modules for the string opengl32sw.dll and it doesn't find anything. (Using the mingw build of opengl32.dll and renaming it to opengl32sw.dll in the application directory doesn't work and it crashes for some reason; only MSVC works.)

Even with all of that, starting the application does throw an error that is actually an application pop-up: Application popup: Topaz Sharpen AI.exe - Entry Point Not Found : The procedure entry point ?GetCurrentContext@egl@@YAPEAXXZ could not be located in the dynamic link library libGLESv2.dll. But after clicking OK the application runs!

I mention all of this because it shows that with some applications it involves a lot of trial and error (and time and frustration) and I don't think reading docs would have helped with this. You can't always trust tools like Dependency Walker or x64dbg.

[pal1000]

I decided to try running Topaz Sharpen AI. I discovered the following:

• It completely ignores opengl32.dll in current directory and only loads opengl32sw.dll from there as you already discovered;
• it bundles an ancient version of Mesa that it can't use (idles with black screen when used);
• it loads both Mesa and the bundled ANGLE library (llibEGL.dll, libGLESv2.dll) at the same time.

The last point is the worst offender as it most likely causes symbols collision at runtime, as both Mesa and ANGLE contain OpenGL ES implementation. You have to get rid of ANGLE before things can work. Remove or rename libGLESv2.dll for that error you still have to go away. MinGW binaries should start working as well if you do it.

I'll implement workarounds for messy programs like this in per application deployment tool. It needs the following improvements to handle mess like this:

• [x] creation of symbolic link named opengl32sw.dll in addition to opengl32.dll (b31cafcc);
• [ ] replace libEGL.dll with libglapi.dll if deploying OpenGL ES;
• [ ] backup non-Mesa copies of llibEGL.dll, libGLES_CM.dll, libGLESv1_CM.dll, libGLESv2.dll and original opengl32sw.dll if exists (warn about overwrite as of 13e86ba8)

As you can guess this is hard. Fortunately not many programs behave so chaotic. I reopened this so I don't forget about it.

[BlohoJo]

It's very nice to see you actually care about making MESA work with as many applications as possible, even badly coded ones. Thanks for the follow up!

I did want to point out, in the case of Topaz Sharpen AI, if I rename/remove libGLESv2.dll, then the application once again crashes with the Qt5Gui.dll crash. So it is apparently needed. Again, I'm running this via VMWare on a system with basically no graphics hardware (the point of using MESA ;).

[pal1000]

Mesa version of libGLESv2.dll is needed if using MSVC package. Mingw package doesn't include that file because it's built the old fashion way to avoid this upstream bug.

[BlohoJo]

I looked into this a little more, and discovered the following (MESA version is 19.3.3 for both mingw and MSVC, and environment variable MESA_GL_VERSION_OVERRIDE=3.3COMPAT is set):

Application directory with just mingw build of opengl32.dll renamed to opengl32sw.dll: Qt5Gui.dll crash

Application directory with mingw build of opengl32.dll renamed to opengl32sw.dll, and libglapi.dll... the one that comes with the MSVC build of MESA (392,704 bytes): Qt5Gui.dll crash

Application directory with mingw build of opengl32.dll renamed to opengl32sw.dll, and libGLESv2.dll... the one that comes installed with Topaz Sharpen AI (2,008,064 bytes): Qt5Gui.dll crash

Application directory with mingw build of opengl32.dll renamed to opengl32sw.dll, and libGLESv2.dll... the one that comes with the MSVC build of MESA (134,144 bytes): Appliction popup message, "Application popup: Topaz Sharpen AI.exe - System Error : The program can't start because libglapi.dll is missing from your computer. Try reinstalling the program to fix this problem." HOWEVER, after clicking OK Topaz Sharpen AI starts and works!

Application directory with mingw build of opengl32.dll renamed to opengl32sw.dll, libGLESv2.dll... the one that comes with the MSVC build of MESA (134,144 bytes), and libglapi.dll... the one that comes with the MSVC build of MESA (392,704 bytes): Appliction popup message, "Application popup: Topaz Sharpen AI.exe - Entry Point Not Found : The procedure entry point ?GetCurrentContext@egl@@YAPEAXXZ could not be located in the dynamic link library libGLESv2.dll." HOWEVER, after clicking OK Topaz Sharpen AI starts and works!

Application directory with just MSVC build of opengl32.dll renamed to opengl32sw.dll: Qt5Gui.dll crash

Application directory with MSVC build of opengl32.dll renamed to opengl32sw.dll, and libglapi.dll... the one that comes with the MSVC build of MESA (392,704 bytes): Qt5Gui.dll crash

Application directory with MSVC build of opengl32.dll renamed to opengl32sw.dll, and libGLESv2.dll... the one that comes installed with Topaz Sharpen AI (2,008,064 bytes): Qt5Gui.dll crash

Application directory with mingw build of opengl32.dll renamed to opengl32sw.dll, and libGLESv2.dll... the one that comes with the MSVC build of MESA (134,144 bytes): Appliction popup message six times, "Application popup: Topaz Sharpen AI.exe - System Error : The program can't start because libglapi.dll is missing from your computer. Try reinstalling the program to fix this problem.", then application popup message four times, "Application popup: Topaz Sharpen AI v1.1.3: Topaz Sharpen AI.exe - System Error : The program can't start because libglapi.dll is missing from your computer. Try reinstalling the program to fix this problem.", then the following non-logged message: https://imgur.com/8B1RVuH. Application closes.

Application directory with mingw build of opengl32.dll renamed to opengl32sw.dll, libGLESv2.dll... the one that comes with the MSVC build of MESA (134,144 bytes), and libglapi.dll... the one that comes with the MSVC build of MESA (392,704 bytes): Appliction popup message, "Application popup: Topaz Sharpen AI.exe - Entry Point Not Found : The procedure entry point ?GetCurrentContext@egl@@YAPEAXXZ could not be located in the dynamic link library libGLESv2.dll." HOWEVER, after clicking OK Topaz Sharpen AI starts and works!

It appears that for some reason, libGLESv2.dll must be present for either the mingw OR the MSVC build of MESA in order for Topaz Sharpen AI to work. Using the mingw build WITHOUT libglapi.dll (which it shouldn't need) will produce an application popup about it being missing for some reason, but the application still runs. The MSVC build of MESA does have to have the libglapi.dll file from its build present or the application won't run.

So the application runs using:

• Mingw build of MESA, opengl32.dll must be renamed to opengl32sw.dll AND libGLESv2.dll from MSVC build of MESA must be in application directory (produces "libglapi.dll is missing" application popup on start, can be ignored)

• Note on the above, if libglapi.dll from MSVC build of MESA is placed in application directory, application popup on start changes to "Entry Point Not Found" message, can be ignored

• MSVC build of MESA, opengl32.dll must be renamed to opengl32sw.dll AND libGLESv2.dll AND libglapi.dll must be in application directory (produces "Entry Point Not Found" application popup on application start, can be ignored)

[pal1000]

MSVC build of MESA

• opengl32sw.dll, libGLESv2.dll and libEGL.dll that come with the application must be renamed/backed-up;
• Mesa opengl32.dll, libglapi.dll and libGLESv2.dll must be in application directory;
• Mesa opengl32.dll must be renamed to opengl32sw.dll,

  MinGW build of MESA

• opengl32sw.dll, libGLESv2.dll and libEGL.dll that come with the application must be renamed/backed-up;
• Mesa opengl32.dll must be in application directory;
• Mesa opengl32.dll must be renamed to opengl32sw.dll,

That's how you avoid all errors.