Electron Windowing Challenges

[captbaritone]

Winamp consists of n windows. The three "main" windows which we already have, but also arbitrarily many additional menus as provided by plugins and other built in features like the browser. Windows in Electron have a number of limitations. Each one is a different HTML document, and they can only talk to each-other via a message protocol. The good news is that our Redux architecture is friendly to a messaging bus approach. However, I see a few real challenges which may make robust Electron support infeasible.

Does anyone see other solutions to these problems?

Visualizing audio in multiple windows

The visualization data is only available in the same context (HTML document) as the actual audio element. For example, we could keep the audio source in the context of the main window, and that would make it easy to have the visualizations work in the main window. However, how would we also support other visualization windows like, the visualizer that can appear in the playlist if it's expanded and the main window is closed?

How do we handle the AVS visualizer window (which I'm hoping we can get done eventually)?

It's possible that we could broadcast the relevant audio data across the message bus, but the the serialization/deserialization requited, I'm skeptical that the latency will be acceptable. Maybe we should try this?

Other challenges

Here are some other challenges which I foresee which are probably possible to solve, but might require some somewhat crazy hacks.

1. Winamp2-js depends upon a huge dynamically generated CSS stylesheet which is generated when the skin is parsed. Do we send this across the message bus and ensure that each window is listening? Possible I suppose.
2. Selecting files from windows that don't own the audio source might be hard. We would have to find some way to serialize the selection, send it over the message bus and then load it from the window that actually owned the audio source.
3. I don't see a solution to the rounded corners added on MacOS.

[durasj]

There is one way to handle all of them if you consider this just an experiment or we don't expect us to do crazy stuff like having the playlist on one end of the screen and player on the other - simply make it as a one window as it is now that would expand when needed. We need to make it transparent anyway because of the skins and also need to support click-through. I know the latter is kinda tricky in the electron, so we could fall back to minimizing on the unsupported platform (till there is a solution). This allows us to use all current logic. Resizing the window dynamically could be possibly harder (especially when moving windows up/to the left), but one way to achieve that would be to maximize it when some window management action starts and then resize to just fit the windows after it ends.

Regarding the other challenges:

1. Shouldn't be that hard (even if we go with multiple windows) as you said.
2. I don't think we need to think about it as "opening in the window". Since there is always one main process, I would propose to handle it in the main (non-renderer) process. Open dialog in the electron returns a list of paths and I don't see the need to use the input type="file" if that's what worries you.
3. Me neither. We could add a tiny margin and implement the transparency, but the windows wouldn't line up. But that would be a way to solve it if they would be one window.

Btw. nice choice of Windows 7 sample song :)

[captbaritone]

Having a transparent window sounds like a great solution, if we can make it work. As you point out, it basically solves all our problems for us. However, from what I've read, getting click-through to work is kinda possible on Windows, and not (yet?) possible on other platforms. Like you said, if people always keep their windows in the shape of a contiguous rectangle, it would be theoretically doable, but if not, we'd have to depend upon robust click-through. From what I read, it's not clear if it's coming or not. Moreover, if people do keep their windows in a strange arrangement, the behavior would simply be confusing, rather than being explicitly in error.

Personally, I'm fine with this being an experiment, but given the enthusiasm I've seen on Twitter for an Electron version, I fear it might be hard to correctly set expectations. I do worry a bit that we'll hit an "uncanny valley" of UI.

Maybe the next step should be to try to get all of Winamp2-js to render in a transparent window that takes up the full screen?

[durasj]

Actually, it should be possible on all platforms. You can see the win.setIgnoreMouseEvents doesn't have any platform label, so I hope it works everywhere. What's different is just a way how to check if the cursor is on the part of our UI or not. But there are some good ideas in the comments so I think it should be feasible. One thing that worries me slightly more is the second bullet in the Frameless Window wiki page: "Transparent windows are not resizable. Setting resizable to true may make a transparent window stop working on some platforms.". I think that's the last thing we need to figure out.

Interesting point with the "uncanny valley", I think there is something in it.

I agree with the next step you mentioned. It needs to be done anyway for transparent skins.

[captbaritone]

@durasj Do you want to try to get transparent windows working in your proof of concept?

[durasj]

@captbaritone Yes I would like, I will get to it on Thursday.

[durasj]

@captbaritone I'm sorry but I wasn't able to get to it just yet because I'm quite swamped. But I remember and keep it in my notes and will get it to it in the next couple days (possibly Monday evening CET).

[captbaritone]

Got it. Thanks for the update!

[durasj]

@captbaritone I experimented a little bit with the transparency, click-through and resizing. Transparency works flawlessly, click-through works well too (but I need to optimize it as much as I can to reduce the overhead) and basic resizing/repositioning works as well.

I think the only tricky part left is how to handle resizing and dragging - I can either keep the window "full-screen" all the time (ugh) and leave all the moving of the windows or switch to full screen only when some resizing or dragging occurs. The hard part of the latter is that I need to reposition windows inside the main window and also the main window itself at once and without any flickering. Do you please have any (other) ideas how to tackle that?

[captbaritone]

Interesting! I wonder if it works x-platform.

Honestly, I'm not that opposed to having it live full time all the time. If click through is buggy, it's going to be bad. If it's not buggy, then having it full screen should be fine.

That said, if it's only a little buggy, then I guess shrinking the window when possible could make it "less bad".

I'd love to play with it myself. Any chance you could push your most recent work?

[durasj]

@captbaritone yes it does. At least on the Windows and Mac OS. I don't have Linux running atm, but all those APIs are cross-platform.

I've made it span the whole screen and enabled back all windows controls. You can see it in transparency branch. We need to figure out few more things:

1. Optimization - currently it isn't very smooth (or as smooth as native ones)
2. Enable dragging across multiple screens - currently works only on one screen which can be bummer in multimonitor setups - probably by implementing some windows control to transfer the window across monitors
3. Fix thumbnail on the windows - shouldn't be that hard as long as we know the position of the main window (that's what is also displayed in the original Winamp)

Please let me know what you think

[durasj]

Btw. @captbaritone can you tell which one is original and which one is yours? :)

[captbaritone]

Amazing work! Just pulled it down, and it's pretty incredible to be able to move the windows around independently! 👏

Multiple screens does sound pretty tough to do. I agree that at the very least it would be nice to move all of them from one window to the other. I think I'm okay with not being able to spread the windows across different monitors. Maybe we could also think about how we can communicate this shortcoming to users.

Regarding optimization, I'm curious what you are seeing that's not running as smoothly. Is it window dragging?

[durasj]

Yes, mostly window dragging. But what I had in mind when I was writing it is that we could use mouseout event and check the transparency only when it leaves elements to reduce the overhead. But I've just realized it can't be done easily since the elements themselves can be transparent. I guess we'll leave it as it is now and someone will come with some idea or electron will start supporting it natively or whatnot.

Regarding the moving of windows to the other screen(s), how do you think we could implement that in the UI? It's pretty straightforward on the backend.

[captbaritone]

I'm not sure about the dragging across windows... maybe something as simple as: the window stick to the edge of the first screen, and has to be pushed a bit farther into the next screen before the change actually happens. When it does cross the threshold, it takes the other windows with it? Basically there's some implied friction/resistance at the boundary.

Just an idea. I'd love to hear others if people have them. One question is: what formation should the non-moving windows have when they move to the new screen? I think ideally they would have the same (or similar) positions to those that they had on the old screen. Obviously we would need to make adjustments if the screen size of the new screen is smaller, but perhaps we could keep it proportional.

[durasj]

Hi, @captbaritone, we've solved the main problems, and I've opened a Multiple displays support #3 in the desktop repo with your suggestion. I guess we can therefore maybe close it here to keep your issues clean?

[captbaritone]

Sounds good. Thanks.