RustAudio / baseview

low-level window system interface for audio plugin UIs
Apache License 2.0
259 stars 55 forks source link

Baseview occasionally crashing host DAW, when used to handle VST plugin windows. #96

Open PanieriLorenzo opened 3 years ago

PanieriLorenzo commented 3 years ago

Description

When adding a new instance of a VST plugin to a DAW, occasionally this causes the DAW to crash on a SEGFAULT, this was narrowed down to happen somewhere within baseview, or one of the dependencies of baseview (possibly glowcoil/raw-gl-context). Notably this seems only to happen after some (or all) instances of the plugin have been closed, and the plugin is added again.

Debugging with gdb revealed the following stacktrace:

#0  0x00007ffc74374298 in rust_eh_personality ()
   from C:\Users\ <omitted full path> \ <name_of_plugin.dll>
#1  0x00007ffcca8d0b54 in ntdll!KiUserCallbackDispatcher () from C:\WINDOWS\SYSTEM32\ntdll.dll
#2  0x0000000000000000 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Context

How to Reproduce

  1. Clone https://github.com/DGriffin91/imgui_baseview_test_vst2
  2. Follow the instructions in the README.md to compile and install the plugin dll into your DAW of choice
  3. Add the plugin to your effect rack
  4. Wait about a minute or more
  5. Remove the plugin from the rack
  6. Wait about a minute or more
  7. Repeat from point 3 until it crashes. Note: the timing might not be relevant. Note: defocussing the DAW window while waiting between steps might have something to do with it Note: adding the plugin to a different effect rack / channel every time might make it crash faster, or maybe it's placebo
nyanpasu64 commented 3 years ago

One idea is to build the library with Address Sanitizer. You can create a file under the project repo, ".cargo/config.toml", with contents:

[build]
rustflags = ["-Z", "sanitizer=address"]        # custom flags to pass to all compiler invocations

Unfortunately rustc doesn't support sanitizers on Windows, despite LLVM and Clang supporting-ish ASAN on Windows (with MSVC ABI, idk about MinGW). Perhaps someone else can test on Linux?