Crash on launch on Windows 7 when locale is zh-HK/zh-MO

[alvinhochun]

OS: Windows 7 x64 (SP1), locale: zh-HK Display: Intel HD 4000 / NVIDIA NVS 5400M (NVIDIA Optimus) Shotcut version: 15.11.03

Well, it just crashes pretty quickly after showing the splash screen. StackHash it says. The exception code is 0xc0000374 (heap corruption)

Running it with WinDbg attached detects the exception. But Shotcut managed to get to the main window if I instruct it to continue running. Don't know if it would properly though. Here is some log with stacktrace:

[snip]
ModLoad: 00000000`68100000 00000000`6815b000   C:\Program Files\Shotcut\SDL.dll
ModLoad: 00000000`0a0b0000 00000000`0a1f1000   C:\Program Files\Shotcut\avfilter-5.dll
ModLoad: 00000000`68e50000 00000000`69026000   C:\Program Files\Shotcut\avformat-56.dll
ModLoad: 00000000`6af50000 00000000`6af78000   C:\Program Files\Shotcut\postproc-53.dll
ModLoad: 00000000`6a0c0000 00000000`6a143000   C:\Program Files\Shotcut\swscale-3.dll
ModLoad: 00000000`6a3c0000 00000000`6a4bd000   C:\Program Files\Shotcut\lib\mlt\libmltcore.dll
ModLoad: 00000000`6c880000 00000000`6c95d000   C:\Program Files\Shotcut\lib\mlt\libmltdecklink.dll
ModLoad: 00000000`63980000 00000000`639ba000   C:\Program Files\Shotcut\lib\mlt\libmltfrei0r.dll
Critical error detected c0000374
(2184.13a0): Break instruction exception - code 80000003 (first chance)
ntdll!RtlUnhandledExceptionFilter+0x29f:
00000000`76d6ff8f cc              int     3
0:000> g
(2184.13a0): Unknown exception - code c0000374 (first chance)
(2184.13a0): Unknown exception - code c0000374 (!!! second chance !!!)
ntdll!RtlUnhandledExceptionFilter+0x2d2:
00000000`76d6ffc2 eb00            jmp     ntdll!RtlUnhandledExceptionFilter+0x2d4 (00000000`76d6ffc4)
0:000> bt
        ^ Operation not supported in current debug session 'bt'
0:000> k
Child-SP          RetAddr           Call Site
00000000`0022cfb0 00000000`76d70606 ntdll!RtlUnhandledExceptionFilter+0x2d2
00000000`0022d080 00000000`76d71812 ntdll!EtwEnumerateProcessRegGuids+0x216
00000000`0022d0b0 00000000`76d734f4 ntdll!RtlQueryProcessLockInformation+0x972
00000000`0022d0e0 00000000`76d0b033 ntdll!RtlLogStackBackTrace+0x444
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Windows\system32\msvcrt.dll - 
00000000`0022d110 000007fe`fd5810c8 ntdll!RtlIsDosDeviceName_U+0x5573
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Program Files\Shotcut\libiconv-2.dll - 
00000000`0022d190 00000000`006f41e9 msvcrt!free+0x1c
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Program Files\Shotcut\libmlt-6.dll - 
00000000`0022d1c0 00000000`69adca6a libiconv_2!libiconv_close+0x9
00000000`0022d1f0 00000000`69adcc62 libmlt_6!mlt_animation_close+0xda
00000000`0022d270 00000000`69ac905a libmlt_6!mlt_properties_from_utf8+0x52
00000000`0022d2d0 00000000`69ac9211 libmlt_6!mlt_properties_parse+0x12a
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Program Files\Shotcut\lib\mlt\libmltfrei0r.dll - 
00000000`0022df20 00000000`63982789 libmlt_6!mlt_properties_load+0x21
00000000`0022df60 00000000`69ad96e0 libmltfrei0r!mlt_register+0x79
00000000`0022e540 00000000`69ad8d9e libmlt_6!mlt_repository_init+0x170
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Program Files\Shotcut\libmlt++-3.dll - 
00000000`0022e5c0 00000000`640021c7 libmlt_6!mlt_factory_init+0x3fe
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for shotcut.exe - 
00000000`0022ea00 00000000`00418474 libmlt___3!ZN3Mlt7Factory4initEPKc+0x17
00000000`0022ea40 00000000`00434fed shotcut+0x18474
00000000`0022eaf0 00000000`0041660f shotcut+0x34fed
00000000`0022ec30 00000000`0042e4b3 shotcut+0x1660f
00000000`0022ec90 00000000`0040f0ef shotcut+0x2e4b3
00000000`0022f190 00000000`00411845 shotcut+0xf0ef
00000000`0022fbc0 00000000`004017e4 shotcut+0x11845
00000000`0022fc00 00000000`004cd6d6 shotcut+0x17e4
00000000`0022fdc0 00000000`004013ed shotcut+0xcd6d6
00000000`0022fe60 00000000`004014fb shotcut+0x13ed
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Windows\system32\kernel32.dll - 
00000000`0022ff30 00000000`76aa5a4d shotcut+0x14fb
00000000`0022ff60 00000000`76cdb831 kernel32!BaseThreadInitThunk+0xd
00000000`0022ff90 00000000`00000000 ntdll!RtlUserThreadStart+0x21

I will try updating display drivers tomorrow if necessary.

Any idea if anything else could be wrong?

shotcut-log.txt on crash:

[Info   ] <Application::Application(int&, char**)> Starting Shotcut version 15.11.03 
[Info   ] <Application::Application(int&, char**)> Windows version 144 
[Info   ] <Application::Application(int&, char**)> number of logical cores = 4 
[Info   ] <Application::Application(int&, char**)> locale = QLocale(Chinese, Default, HongKong) 
[Info   ] <Application::Application(int&, char**)> install dir = "C:/Program Files/Shotcut" 
[Debug  ] <> begin
[Debug  ] <> end
[Debug  ] <> begin
[Debug  ] <> begin

[bmatherly]

Video card driver seems like a good place to start.

Maybe also try English Locale.

[ddennedy]

I am not keeping open bugs on general launch failures and system incompatibilities. It works for me and many others. I know it does not work for everybody. Either it works for you, or it doesn't, and maybe you will debug it or not.

[alvinhochun]

Well, I guess I might be doing that too. If the latest display driver doesn't fix it, I'd better just try git HEAD, right? Guess I would be downloading the latest Qt later and freaking run out of disk space...

Is there a release with debugging symbols though?

[ddennedy]

There is no release with debugging symbols, and it is not trivial to setup a development environment for Shotcut especially on Windows: http://www.shotcut.org/bin/view/Shotcut/SetupWindowsDev

You do not need to use a specific version of Qt Creator (or that IDE), but you will need to configure a "build kit" to point to a specific compiler version and Qt SDK - not just any will do. The Shotcut SDK alone (not including Qt, MinGW, and Qt Creator) is 761 MB.

If you want to proceed, e-mail me for the Shotcut SDK download link. I will not post it here because I do not want to make it easy for just anyone to try, bug me with questions, make malware-infested knock-offs, etc. You can easily find my e-mail address in the comment headers of the source code.

[alvinhochun]

And well... looks like I found the cause... it is the locale, duh... It's even hinted on the stack trace, that iconv_close is related to the crash.

Having the locale (formatting, not display language or non-unicode program codepage) set to zh-HK (or also zh-MO, but I am not in Macau) makes Shotcut crash, but zh-TW doesn't. Which is weird since all three usually use Big-5 (codepage 950). (Well... arguably Big5-HKSCS is also a possibility, but Windows doesn't use it by default I believe.)

I have never used iconv though, so it probably will take me some time to look into this. Perhaps I should find where iconv is used. The stack trace points to libmlt, so I guess I should start there?

[ddennedy]

See here: https://github.com/mltframework/mlt/blob/master/src/win32/win32.c#L69

Probably cd is invalid pointer because encoding was not accepted, and iconv_close() does not handle invalid iconv_t. This function is only used by the function directly below it; so you can see what/how encoding is being passed. Certainly, I can add a check if "cd" is valid before iconv_close(). Then, I can give you a new build to test tomorrow. Look over mlt_properties_from_utf8() and let me know if you see something else suspicious for your locale and codepage.

[alvinhochun]

Ok, I think I found the code.

https://github.com/mltframework/mlt/blob/master/src/win32/win32.c#L74 which is called by https://github.com/mltframework/mlt/blob/master/src/win32/win32.c#L95

It seems that it is calling iconv_open with A.R..950 (those are actual dots with char code 2e) as the first argument (encoding). A.R. seems to be a substring of Hong Kong S.A.R, which makes a lot of sense, since in mlt_properties_from_utf8 it attempts to take the substring after the first dot.

Here's what I think... if you want to get the system ANSI codepage (non-unicode program codepage), you should better just use GetACP. Also, the user locale does not necessarily match the ANSI codepage since it can be configured differently.

I guess there might also be some bugs in libiconv too which made it failed to reject the erroneous encoding.

[ddennedy]

I will change to use GetACP() and test "cd" before iconv_close() and give you a overnight build to test tomorrow. Re-opening since you are helping and this might be resolvable. Thanks

[alvinhochun]

Actually, when I think about it more carefully, using setlocale to get the codepage could be correct in some cases. It seems that the setlocale locale affects C library string functions while GetACP gets the codepage which affects Windows API calls. Depending on the usage, one might be better than the other. Also, user locale, default user locale and system locale in Windows are really confusing.

Can you tell what mlt_properties_from_utf8 is used for? Is it for data in a file with ANSI encoding or something else?

Though I guess you can go on with the GetACP build and I will test it after work.

Edit: _getmbcp also seems to be a possibility.

[ddennedy]

The purpose of this function is to convert UTF-8 strings as standardized by MLT to a filename that can be used by other libraries such as libavformat and libxml2 that expect file names to be in an encoding compatible with the Windows file system. setlocale() as used here is only used to lookup the system locale - not to set it for the C lib functions.

[alvinhochun]

According to MSDN docs for fopen

By default, a narrow filename string is interpreted using the ANSI codepage (CP_ACP).

In this case, GetACP seems to be the way to go.

[ddennedy]

Here is the MLT commit: https://github.com/mltframework/mlt/commit/a26deb7e15294ca40b17e5dab780da9d0f065ba9

A new build will be available in about 12 hours.

[alvinhochun]

Thanks, it started without crashing.