Use Godot for the user interface and input

[TechnoPorg]

Enhancement Summary

I propose that LMMS should switch to using Godot, rather than QT, for its user interface and input handling. Possibly audio as well, although I'm not sure if Godot's sound system would work well for what LMMS needs. The backend code would become a GDExtension plugin. I would recommend going with 4.0.dev right away, even though it's not stable, to avoid porting everything from 3.x to 4.0 later.

Justification

There are several reasons why I believe this to be a good idea:

• QT is massive. v5.15.12 for Windows, which I have installed currently, is 2.28GB. Godot 4.0.dev (https://github.com/godotengine/godot/commit/77721b35ba21f2e7e8bb42cf415fccb018517843) with debug symbols is only 193MB.
• With Godot the UI could be edited with an editor, rather than through code.
• UI/UX elements could be handled in a GDScript, a higher-level language designed specifically for Godot, if desired. QT does have QML, so this point on its own isn't a win for either option.
• However, Godot makes application plugins incredibly simple. All you have to do is load a .pck containing a plugin scene, instance said scene, and the plugin is running with full access to the UI elements. This plugin would likely be written in the aforementioned GDScript, which is easy to use.
• It would fix the situation where LMMS has to be installed on Windows in order for the executable to have access to the assets.
• It would be a logical point to implement a whole bunch of big UI changes, notably #1911. If this proposal is accepted I'll gladly try implementing it and overhauling the UI.

  Mockup

  Sorry to inflict my lack of art skill on your eyes :)

[Marzin-bot]

Create a simple website which I think has never been difficult.

[SeleDreams]

godot wasn't suggested because it was "less complicated" that was what some users thought and it's subjective but more because of its advantages of being lightweight, having a wide support of languages, having a nice built in UI system, and being modular

also it doesn't require many dependencies from what I remember

[SeleDreams]

the other advantage is godot is another open source project so both LMMS and Godot contributors could help both projects at once

[Marzin-bot]

Electron is licensed under MIT. Electron is based on chrome which also has an open source license. chrome is based on the WebKit web engine which has also become open source.

[SeleDreams]

electron is open source, but it's more company based than community based, godot is entirely supported by volunteers like LMMS

[Marzin-bot]

Electron is basically a web browser that exposes functionality beyond the sandbox. Nothing obliges to go through Electron whereas one can directly use WebKit which is "simple" to use in C ++. But personally, I see more interest in using Electron directly.

[wwderw]

Electron exposes features that are not directly available in web browsers, which allows direct communication with the user's operating system. So no sandbox problem

Thru Node, yes. Webview also allows for atleast some of this as well. At least the last time that I tried messing with this type of SDK. There are usually some module that one can import to do your own server to use via webview to take the place what node does as well.

As long as it is using some type of browser (be it webview or Electron) it is still going to not quite be the same as traditional apps. The browser experience is just following a different paradigm. I think that they could do better to help bridge that experience, but that remains to be seen if the powers that be go down that route.

Now using something like Raygui (although that may not be feature complete for a complex app like this) with C/C++ would yield better resource usage compared to Godot. Unless minimize how much is done via GDScript and do the logic totally in language like C++.

[Marzin-bot]

I add that there is a C ++ and electron interface without depending on WASM.

[Marzin-bot]

So by itself a large part of the LMMS kernel will not need to be rewritten / adapted.

[SeleDreams]

Electron exposes features that are not directly available in web browsers, which allows direct communication with the user's operating system. So no sandbox problem

Thru Node, yes. Webkit also allows for atleast some of this as well. At least the last time that I tried messing with this type of SDK. There are usually some module that one can import to do your own server to use via webkit to take the place what node does as well.

As long as it is using some type of browser (be it webkit or Electron) it is still going to not quite be the same as traditional apps. The browser experience is just following a different paradigm. I think that they could do better to help bridge that experience, but that remains to be seen if the powers that be go down that route.

Now using something like Raygui (although that may not be feature complete for a complex app like this) with C/C++ would yield better resource usage compared to Godot. Unless minimize how much is done via GDScript and do the logic totally in language like C++.

I have just a minimal experience with raylib stuff, always had some interest but never the time to look at it. I dunno if its UI system is as good as godot's one tho

[rdrpenguin04]

I do in fact have legitimate uses for Electron; primarily, I'm making an IDE for a school-related project, and Electron has a good history with small IDEs.

Electron is powerful, but it is slow, and I have seen that firsthand. Real-time audio synthesis needs maximum speed with minimum compromises, and that is something a browser engine can not provide, not with the JS engine or even the DOM engine.

[Marzin-bot]

There are also breaks that can occur between major versions of Godot (3 / 4.x). Something that is severely weakened by the Web standards for Electron.

[Marzin-bot]

It is possible to use your own AUDIO C ++ API in addition to making sure not to import JS and API classes that are not used. Suddenly no more problem of slowness.

[rdrpenguin04]

LMMS has been dealing with Qt breaks for quite a while; also, Godot breaks are so minor that they can get away with putting documentation for 4.x into the 3.x docs... No, they can't get away with it, but the language is apparently similar enough that it'll stand up. My point is that breaking changes won't be an issue. Secondly, you seem to have forgotten that the DOM engine is just as much of a problem as the JS engine is, and it's not something you can simply circumvent.

[Marzin-bot]

I don't see where the problem is with the DOM engine. In Godot, the UI is also managed as a tree like the DOM.

[Marzin-bot]

When interacting with Visual Studio and Discord, I don't have a problem with slowness anyway.

[wwderw]

I have just a minimal experience with raylib stuff, always had some interest but never the time to look at it. I dunno if its UI system is as good as godot's one tho

No, Godot's predefined widgets/nodes is better, in my opinion, compared to Raygui. Simple apps, Raygui does well.

There is, to tangentially go with this project uses, a program reminiscent of JFXR (JFXR being browser based), that was made using Raylib, by the creator of Raylib.

I think immediate mode gui gets a bad rap, and I personally like them and done right, not that much of a performance hit compared to the traditional cross platform toolkits.

I've dabbled using AudioNodes(Electron based), and while I didn't have a problem using it and I did like its feature set, it did like its resources.

[rdrpenguin04]

I don't see where the problem is with the DOM engine. In Godot, the UI is also managed as a tree like the DOM.

The tree isn't the problem, the problem is the layout and styling required for a DOM to be rendered. Godot is optimized to handle simple information simply. Electron is optimized to handle complex information in whatever way necessary, which makes it significantly slower.

When interacting with Visual Studio and Discord, I don't have a problem with slowness anyway.

Just because you have no issues, that doesn't mean someone else wouldn't have the same problems. I have often been on the side of slower than average, so I have plenty of experience in this area.

[Veratil]

I believe you're all missing the point of what we're trying to get LMMS to. If we get the core split from Qt then we have a DAW backend as liblmms essentially. The frontend after that can be in Qt, Electron, or Godot, or anything else really.

That being said, I believe we'll keep the frontend as Qt for now since most of the work for it is there already and updating is fairly quick to new Qt versions. What we get out of the core separation though is the ability for all of you to create these frontends to expand the reach of LMMS. Electron could allow people to run a version of LMMS in the browser on the go. Qt would be the desktop version. Godot could be an alternative and possibly after some time and learning the default desktop option. It would definitely be able to draw faster using the GPU than Qt afaik. :)

[wwderw]

When interacting with Visual Studio and Discord, I don't have a problem with slowness anyway.

Depending on the specs of your machine, may never see an issue. Doesn't mean the individual program isn't resource hungry though.

[rdrpenguin04]

I believe you're all missing the point of what we're trying to get LMMS to. If we get the core split from Qt then we have a DAW backend as liblmms essentially. The frontend after that can be in Qt, Electron, or Godot, or anything else really.

For the record, I did address that:

Nevertheless, once we abstract the UI out, it would be possible.

[radlotus]

Guys, I just wanna say that I really like the very easy piano roll and pattern system, and the browser on the left. Out of Ableton, Reaper, Ardour and FL Studio I like LMMS the most. It's very easy to write melodies and work with patterns and introduce complexities and variations. It's also very easy to use the computer keyboard as a MIDI keyboard, and i really like that LMMS catches the key presses even when you are inside a plugin. FL Studio is the closest, but i really hate the convoluted pattern system and the floating windows management which i think its worse than LMMS. I don't mind the floating windows in LMMS. I think LMMS's interface shouldnt deviate too much. I think its fine right now just as it is that's why its been successful even with so many drawbacks, that mostly consist of absence of some features. I don't want LMMS to become in terms of UI like Reaper, Ableton, Zrythm, Waveform, Cakewalk, Studio One, Pro Tools which share very similar ui which i don't like, except maybe for Reaper. I like Reaper the most out of the listed. It's easy actually and very stable and functional, but ultimately I like LMMS more because its easier on my adhd brain and the music i make is heavily based on complex beats and melodies which i write by hand in the piano roll.

[aaronfranke]

Disclaimer: I actively contribute to Godot Engine.

Godot is indeed a really good GUI framework. GDScript, Control nodes, and Godot's visual editor makes it super easy. Godot also has a lighter footprint than most toolkits, even Electron. Note that Godot has one major disadvantage vs others which is that it requires OpenGL or Vulkan to render things, so it won't work well in VMs or really old systems.

I would recommend going with 4.0.dev right away, even though it's not stable, to avoid porting everything from 3.x to 4.0 later.

4.0 dev is very unstable right now, so I'd be hesitant to suggest this, but maybe it's a good choice in the long term. Considering this issue is still being debated and there needs to be core refactoring done to split the front-end from the back-end, it may be better to wait.

although I can imagine a few performance issues arising from the fact that these usually run on a constantly iterating game loop instead of updating on events only. But maybe that's not too much of a concern.

Godot has mechanisms for event-driven programming including signals and notifications, it's completely optional to use every-frame logic like the _process function. Also note that Godot has a low processor mode for not updating when nothing changes on the screen, which the editor uses. However when something does need to update, Godot does not have partial redraws, so it has to redraw everything.

And Qt is used for so much more than just GUI. It provides memory cleanup and the signal/slot mechanism. Plus some other silly stuff like XML, I/O, timers, smart pointers, threads, vectors, string translation and debug messages. All of this can be done in some other way but someone's got to do it.

I think Godot provides all of this (not sure about smart pointers).

I haven't found any supports in Godot even for multi-window.

Godot 4.0 will have support for multiple windows.

Additionally Godot already has a mechanism in which you can build the engine with a selection of components in order to reduce the executable size. Currently there is no mode for the UI only AFAIK but that can possibly be added.

I plan to make an unofficial GUI-optimized fork of Godot that will be released alongside Godot 4.0, but I haven't made the repository/branch yet because I am trying to get most of the heavy lifting work upstreamed into Godot.

[Yam76]

So if I understand correctly, this project's future work is to extract lmms from Qt into liblmms, then try out Godot 4.0 when it arrives, and if it works well, great, if it doesn't, stick with Qt.

[Veratil]

So if I understand correctly, this project's future work is to extract lmms from Qt into liblmms, then try out Godot 4.0 when it arrives, and if it works well, great, if it doesn't, stick with Qt.

The plan is to split the core and the frontend, period. This allows us to create frontends for anything: mobile, TUI, a MIDI tracker (with expanded capabilities of a full DAW), etc. We could have a frontend with Qt and have a frontend with Godot. We don't have to remove one to have the other with the split setup.

[Yam76]

Will there still be an official frontend? I'm mentioning this because I am concerned it will turn into a massive mess otherwise. Poor new user experience, messy and abandoned frontends, a growing disconnect between what backend devs write and what frontend devs want, etc.

[Veratil]

Will there still be an official frontend? I'm mentioning this because I am concerned it will turn into a massive mess otherwise. Poor new user experience, messy and abandoned frontends, a growing disconnect between what backend devs write and what frontend devs want, etc.

Yes, we'll always have an officially supported frontend for desktop. Which framework ends up being is up to community consensus and ease of work. Qt will be the main one for now, and if Godot proves to be easier to maintain and extend as needed then it's entirely possible to move to it. A good argument will be needed to completely switch to a new framework though since Qt is dug in deep right now. And remember it's not just me you have to convince. :) (personally I'm a fan of Godot and have been wanting to learn it, so this would be a good excuse to finally jump in)

As for messy and abandoned frontends, I will always fight against having multiple frontends in the main repo unless they are going to be actively supported. That being said, we're still a good ways away from getting to this point, but a well thought out plan is a good thing.

[Spekular]

Locking this for now, because I don't think it's particularly productive to continue this discussion right now. Until the core and UI are properly separated a Godot UI isn't happening, and once they are:

• The main frontend should probably be decided via a discussion that doesn't start at the conclusion. Think "what framework should we use?", not "why should we use framework x?"
• Anyone who wants to can create a separate frontend with Godot / Electron / Vulkan / Etch-a-sketch

I also feel that these kinds of issues generate lots of:

• Unfounded excitement: "it'll be awesome when [my specific vision of the future] happens"
• Unfounded worry: "please don't remove the piano roll when switching frameworks"
• Conversation rehashing things that have already been decided or thoroughly discussed: "what if the core was separate from the frontend?"

[tresf]

Locking this for now, because I don't think it's particularly productive to continue this discussion right now. Until the core and UI are properly separated a Godot UI isn't happening, and once they are:

Might be a better candidate for Discussions. (Do people use those?) https://github.com/LMMS/lmms/discussions. I feel that there's some good information here for people looking to prototype something, especially when we have active framework developers chiming in. 🍻

[tresf]

Doubting whether Godot is a good choice for such a big application... I was eagerly looking for a flexible UI layout with the docking panels, like Visual Studio and Eclipse and also mentioned in an issue 6 years ago here: #1911 (comment) I consider the docking should be a basic UI feature for big applications - even VC++6.0 did this better than most of others - and with Qt this mightn't be too difficult since we have QDockWidget

Interestingly, if you read the fourth comment down from that on that same bug report, you have the maintainer of ZynAddSubFX making the following statement about Qt:

Well, QT pretty much anything is very much unsafe to use anywhere near the realtime side of things.

... a comment about whether or not the signals and slots system offered by Qt is a suitable long-term solution for core/ui communication.