frang75 / nappgui_src

SDK for building cross-platform desktop apps in ANSI-C
https://www.nappgui.com
MIT License
503 stars 50 forks source link

(WIN32) i_get_window incorrectly assumes that GetWindowLongPtr will return an OSWindow #156

Open colugomusic opened 1 month ago

colugomusic commented 1 month ago

I am using view_native to get the native window handle. I need to do this as I am using nappgui to launch audio plugin editor windows (e.g. VST/CLAP), and I need to pass the native window handle to the plugins to allow them to embed their own UI into the OS window.

If the audio plugin happens to create their own window handles internally then these foreign windows will end up being processed by nappgui's message loop.

The function i_IsDialogMessage will then call i_get_window with a HWND which is unknown to nappgui, but nappgui assumes that, if GetWindowLongPtr returns non-null, then it must be an OSWindow. This will result in a crash.

I would recommend instead keep a list or map of HWNDs which are known to be managed by the nappgui framework, and ignoring any "foreign" window handles.

At the moment I am simply commenting out the call to i_IsDialogMessage in my fork of nappgui but it would be nice to have a better solution.

I haven't tested this on Linux and macOS yet but I assume this is a problem specific to Windows' message processing.

frang75 commented 1 month ago

Hi @colugomusic

At the moment, NAppGUI is designed to create its own runloop. However, I would like to add the support you need for your project. I would need some more context:

colugomusic commented 1 month ago
  • Does your application provide an external runloop (main/WinMain)? Does it use NAppGUI's osmain?

I don't use my own external loop. I use nappgui's osmain_sync macro.

  • Do you need to create a "blank" window with NAppGUI to be "filled" by an external library?

Yes I am mainly just using nappgui as a cross-platform solution to open up windows. All I need is this:

struct device_ui {
    View* view;
    Layout* layout;
    Panel* panel;
    Window* window;
};

I then use view_native to get the native window handle which is passed to the external library (the audio plugin) so that it can attach its own UI to the HWND. I have only done this on Windows so far but this much appears to work, unless the plugin happens to create more of its own HWNDs which will lead to the crash I described.

nappgui is possibly too heavy of a library for what I need (simply a way to open a window) but it seems like the best thing I have found so far. Alternatives I have tried are choc's "DesktopWindow" class (https://github.com/Tracktion/choc/blob/main/gui/choc_DesktopWindow.h) and the "CrossWindow" project (https://github.com/alaingalvan/CrossWindow). But nappgui is also nice in that I can use it to also create debugging controls if I want to.

  • Do you need to create a "full" window with NAppGUI, but managed by an external runloop?

I'm not sure what the difference is between what you call a "blank" window and a "full" window. But I don't use an external message loop for handling the windows. I just use the osmain_sync macro. Do you think I should I be writing my own message loop instead for this use case?

  • Can you provide a complete simple example? A "Hello VST/CLAP" so I can compile it and integrate support in a cross-platform and stable way.

This isn't very easy for me to provide right now as audio plugin hosts are quite complex. You could have a look at the project I am working on here: https://github.com/colugomusic/scuff (The CLAP audio plugin host is partially implemented in the "sbox" subproject.) In the future I might extend the "test-host" subproject into a very simple plugin host but at the moment it's not there yet.

By the way, instead of keeping track of all "known" HWND handles there is another approach which I have tested which appears to work.

I found every call to SetWindowLongPtr(hwnd, GWLP_USERDATA, ...) and added a second call to SetWindowLongPtr(hwnd, DWLP_USER, (LONG_PTR)(1234567)).

1234567 is just a random magic number that can then be used to check if a HWND came from nappgui. Then in i_get_window I just use GetWindowLongPtr(hwnd, DWLP_USER) to check if the value is equal to the magic number.

frang75 commented 1 month ago

Do you have a Discord user?

colugomusic commented 1 month ago

Do you have a Discord user?

Yes it is .colugo

frang75 commented 1 month ago

Related with this issue: https://github.com/frang75/nappgui_src/issues/100

Is the opposite case, but both will be included in NAppGUI 1.5.0 release