Fixing VST plug-ins disappearing when resized

[bluecataudio]

Resizing VST editors when loaded in OBS Studio on Windows most often result in actual plug-in editor to disappear, and requires the window to be reopened. This change fixes the issue in most cases (except when the size of the plug-in decreases).

The original code directly uses the native window instead of the Qt container widget (which is not recommended in the Qt documentation), which results in resizing issues. Using the Qt container widget to resize the window instead fixes the problem, at least for some cases.

The remaining issue when size decreases is probably a problem with Qt and the specifics of the WindowContainer widget. Re-increasing the size of the plug-in editor or reopening the window fixes the problem.

[WizardCM]

Are there specific VSTs that this is reproducible with?

[bluecataudio]

The issue happens with all our plug-ins on Windows. You can easily reproduce the issue by installing any of our free plug-ins:

https://www.bluecataudio.com/Products/Category_0_Freeware/

Once the plug-in has been loaded in OBS, click on the zoom icon (magnifying glass) in the tool bar and select a different zoom level. The resizing callback is called in obs-vst but the window is not resized despite using the appropriate values, and the plug-in editor disappears.

The proposed fix is closer to the obs-vst MacOS source code and avoids direct calls to the native window.

[WizardCM]

Thank you for the additional info.

I have tested this PR, and while resizing to a larger size (100% -> 150%) works correctly, resizing back down (150% -> 100%) it seems to bug out. Tested with BC Fee Amp.

Screenshot:

Resizing down to smaller than wanted (90%) then back up does fix it.

[bluecataudio]

Yes that's indeed what I noticed, as explained in the initial PR post. My guess is that it is a bug in Qt (or in the way this particular Qt widget is used), but I have not had much time digging into the QT source code. They are probably doing some differential size updates and fail doing so when the diff is negative or something like that. The behavior of this widget is pretty strange to me, and seems to be very sensitive to the order of function calls (looking at the MacOS code for the PluginEditor is scary!).

Resizing back to a larger size makes if fine again. so it's not ideal but still better than without the fix. I have not tried with many other plug-ins yet though.

[WizardCM]

The sizing issue when shrinking occurs because QWindow::fromWinId creates a QWindow, and it is having the wrong size set. Qt is just supposed to handle this automatically.

I've pushed a solution that seems to work for me - removing the widget->resize() call in handleResizeRequest. Please try it with my changes, and if all goes well we can squash & merge.

I'm not 100% sure why this solution works, though.

[bluecataudio]

With your fix, re-opening a session with a plug-in inside sometimes has the resizing issue.

So I have committed another change that definitely fixes all cases and got rid of the unnecessary QWidget field that I originally added.

I think I now understand how this Qt widget works: 1/ it has to be initialized with the right size upon creation. Then all subsequent size changes should be done on the child widget only, and it will manage things for you properly.

Thanks for your time on this! Glad to see this working all fine!

[bluecataudio]

BTW I am sorry for the formatting issues. I am not sure why this happens (Visual Studio automatically used the ClangFormat rules to format the files).

[WizardCM]

No worries about formatting - VS 2019 should have the correct version of clang-format, but if you're using something different it may misbehave.

Verified with your changes that things are much happier.

I did notice in the original code that the dynamic resizing was intended for Voxengo SPAN so I've done some additional testing with that. It seems resizing to a smaller size works perfectly (using the resize grip), but resizing to a larger size than the original it does not resize the QWindow correctly. Your original widget->resize() works correctly in this case. That's annoying.

[WizardCM]

Alright, I have a potential solution to handle SPAN's particular case. Thoughts?

        effect->dispatcher(effect, effEditGetRect, 0, 0, &rec, 0);

        if (rec) {
-               resize(rec->right - rec->left, rec->bottom - rec->top);
+               int width = rec->right - rec->left;
+               int height = rec->bottom - rec->top;
+               resize(width, height);
+               if (widget->width() < width || widget->height() < height) {
+                       widget->resize(rec->right - rec->left, rec->bottom - rec->top);
+               }
+
        }
 }

This does reintroduce the previous widget, but should smoothly handle more situations. Unfortunately this code does unnecessarily trigger with Blue Cat plugins, but shouldn't make things worse - not to mention it's a single resize call.

[bluecataudio]

I am not sure why, but with this change, I get a size issue again when reopening an existing session:

• add the Free Amp plug-in.
• Zoom the GUI to 200%.
• Close OBS
• Reopen OBS => the size of the plug-in window is fine, but the editor is partially hidden. It is still better than the original code, but not 100% perfect.

My guess here (but I have no idea if it makes sense) is that the Qt widget works by computing size differences. And when you resize the window in the Voxengo case, you get multiple resize messages in a row before the QT widget has actually been able to apply the size. That's a pretty odd behavior.

[WizardCM]

I have a feeling you are correct. I'd be personally OK merging your PR as-is (without my SPAN-specific fix) and figuring out a separate solution for SPAN at a later date, as even without your PR it has the same issue of going blank on resize, and this is better than that. Sound good?

[bluecataudio]

Sounds good to me, for sure! If it has an issue anyway, I guess it might be another problem, then.

[bluecataudio]

Thanks a lot!

[WizardCM]

I come bearing frustrating news - we have discovered a VST that causes a crash deep in Qt/Windows with this fix. I plan on digging into potential workarounds this weekend, but wanted to document my findings first:

1. The VST in question is mvMeter2 (free download on that page)
2. The crash occurs when either resizing via the grip or using the GUI Scale option it provides
3. Disabling the below line fixes the crash

https://github.com/obsproject/obs-vst/blob/106868e730ad7c34fbb7d25b6100dc36d1c9f147/win/EditorWidget-win.cpp#L65

[WizardCM]

@bluecataudio I pushed another change to master fix the above issue which has also fixed the Voxengo SPAN resize bug. Please test it and let me know if it works for your use case. https://github.com/obsproject/obs-vst/commit/56168fdff9

[bluecataudio]

It works perfectly now, thanks!