search

thestk / rtaudio

A set of C++ classes that provide a common API for realtime audio input/output across Linux (native ALSA, JACK, PulseAudio and OSS), Macintosh OS X (CoreAudio and JACK), and Windows (DirectSound, ASIO, and WASAPI) operating systems.
Other
1.51k stars 322 forks source link

Fix WASAPI backend loading AVRT.dll with UNICODE defined #292

Closed nyanpasu64 closed 3 years ago

nyanpasu64 commented 3 years ago

Previously, RtAudio casted a char-based string literal to a TCHAR const*, and passed it into LoadLibrary (a #define). With UNICODE defined, TCHAR is wchar_t and LoadLibrary maps to LoadLibraryW (which expects UTF-16 input). Windows interprets the string literal as UTF-16 and tries loading 噁呒搮汬.DLL instead. Even worse, this performs an out-of-bounds read because when loading UTF-16 null-terminated strings, Windows looks for two consecutive null bytes. If the string literal's null terminator isn't followed by a second null byte in the program's memory layout, Windows might read even more gibberish and try to interpret it as UTF-16. (This is probably harmless but still bad.)

This PR changes it to unconditionally call LoadLibraryW and pass in a UTF-16 string literal. My guess is that this is slightly more efficient than calling LoadLibraryA (which I guess allocates a new UTF-16 string, calls a locale-to-UTF-16 conversion routine, and passes the UTF-16 string to LoadLibraryW).

(Previously searching for AvrtDll would always fail on UNICODE builds. Now that it succeeds, I'm not certain that this won't expose misbehavior in the if ( AvrtDll ) branch. But hopefully the branch works fine, and hopefully it's been sufficiently tested in non-UNICODE builds. I tried it in my app, and it didn't crash or anything.)

Fixes #291.

MarcusTomlinson commented 3 years ago

LGTM! And thanks for the detailed description!