LMMS redesign concept

[budislav]

(Click to enlarge)

---

Hi all, My previous work for Zynaddsubfx has same purpose to dock all windows in one (link is dead) http://budislavtvp.deviantart.com/art/ZynAddSubFX-UI-Concept-2014-455890191

It is in development phase > https://github.com/fundamental/zyn-ui-two

Link to full concept: http://budislavtvp.deviantart.com/art/LMMS-UI-UX-Design-Concept-Single-Window-523696539 https://www.behance.net/gallery/38503021/LMMS-UIUX-Design-Concept-Single-Window-Interface https://www.behance.net/gallery/194917259/LMMS-Redesign-Concept

Full version (mirror): full version

[oeai]

Looks very good, you can add pics right here, maybe just not all for ui and zyn

[musikBear]

stunning! ..but i am feeling a chill, cause i have spent ...absurd time learning zyn's current UI sigh :hamster:

[BaraMGB]

For your (awesome!) LMMS-UI-Concept I fear this requires a complete rewrite of LMMS's GUI. How do you handle the windows? (switching in fullscreen per button? dragging the corners for sizing?)

Greetz Bara

[tresf]

For your (awesome!) LMMS-UI-Concept I fear this requires a complete rewrite of LMMS's GUI. How do you handle the windows? (switching in fullscreen per button? dragging the corners for sizing?)

I can't speak on behalf of the author, but most DAWs use a collapse/show method and the designated panes are just reused depending on which widget is shown.

you can add pics right here, maybe just not all for ui and zyn

Done. :)

@budislav,

Very interested to know the possibility of this. When you worked with the Zyn team, how involved with the code were you? Did you have to fork the repository? Did they merge the change into base? Last I checked, Zyn wasn't fully Qt compatible but your library seems to make heavy use of Qt and QML (right?)

Tagging @RebeccaDeField @pgiblock

Much overlap with:

880 #1839

[tresf]

I am just UI/UX designer so I can't help with coding.

:+1:

everything you seen is work by Mark McCurry (fundamental)

Thanks for clarification. Yeah, Mark shows up here occasionally. I'm sure his efforts could probably be extended for LMMS as well.

As far as a singleton window, we likely have thousands of lines to clean-up to make this a possibility, but your mockup is delightful and we're likely to use it as a reference point when discussing major UI changes in the future, thank you! :+1:

[tresf]

@fundamental what type of overhaul are we looking at here if we were to adopt this new UI? I assume we'd have to port over pretty much every single widget, but how extensible is this opengl/QML for other projects?

I'll quote the readme for others interested...

• Custom nanovg (opengl 2.0) based rendering
• Custom constraint based layout
• QML based structure/event handling

Progress

As of so far a whole bunch of widget rendering code has been written. A basic horizontal weighted/constrained pack layout has been created (in C) and two simpler pack algorithms have been made in qml. QML/nanovg integration has been establed and it appears to be relatively bug free (though perhaps somewhat inefficient). Some mouse handling has been done. A variety of other widgets have received their design on paper. To test some UI performance/integration a simple equalizer or envelop app might be created.

[budislav]

"your mockup is delightful and we're likely to use it as a reference point when discussing major UI changes in the future, thank you!"

I am glad to hear that, I hope this would be very helpful. I'm tired of floating windows :)

[tresf]

I'm tired of floating windows :)

Yes, many of us all are. Per #519, #765, #1406

And then there's #387, #1185, #1357, #520, #735, #1755, #1849, #1843

Just worked on a track last night and had to make use of the somewhat hidden Qt feature "Always on top" in the left corner of every sub-window. :+1:

[budislav]

"Just worked on a track last night and had to make use of the somewhat hidden Qt feature "Always on top" in the left corner of every sub-window."

That's what I'm talking about :+1: I will upload one mockup that shows how it looks on full HD res.

[tresf]

That's what I'm talking about :+1:

@budislav P.S., you can use the > to quote a messsage. :)

[softrabbit]

Let's say I have a dual screen setup. Now I can stretch my LMMS window to cover both screens and place the subwindows where I like. In which way would the proposed concept be an improvement in that case?

[budislav]

@tresf

P.S., you can use the > to quote a messsage. :)

Thanks, didn't know that :)

[tresf]

Thanks, didn't know that :)

Sorry, it has to be the first character in order to work properly. :+1:

dual screen setup

Yeah, I was wondering this as well.... If the other usability improvements happen that is certainly an obstacle I'd like to have. My recommendation (even in our current design) would be to have unlimited container windows. This could get ugly for singleton objects though (which we have some other open items on already).

[tresf]

@softrabbit looks like we're not the only DAW with this dilemma... Ableton, FLStudio, etc.

[budislav]

@softrabbit I've been thinking about dual monitor but there probably should be some options in preferences just for LMMS to know if you want to use second monitor. I need to think about this more.

[budislav]

One solution will be to undock desired section and to move it on second monitor which have container than dock it automatically. Now there should be some kind of pattern for each section. Browser should be docked only to left or right, and instrument and effect rack should be always on bottom, because it have fixed height.

[unfa]

Seriously, I'm gonna put out some cash for implementing this.

[fundamental]

On 03-31, Tres Finocchiaro wrote:

what type of overhaul are we looking at here if we were to adopt this new UI? I assume we'd have to port over pretty much every single widget, but how extensible is this opengl/QML for other projects?

At the moment the goal is to just get this to work for zyn, but I'd imagine other projects could adapt the code. The layout routines seem like they should have no problem working in other qml projects (and IMO QML really should have such algorithms bundled by default).

The draw routines should be easily swapped out, as most of them are contained within draw.{c,h} (it has really driven me nuts how many toolkits make it so hard to restyle things by spreading out the draw code over so much space).

The mouse interactions should be pretty much identical for many of the widgets if they're reused.

The bindings of the widgets to the engine will need to change as once I reach that stage they're going to be 99% OSC message driven with some sort of caching backend.

So, there will be some mismatch, but nothing that would prevent another project from reusing components.

Given the current rate of development, there's likely a year to go before I get the current replacement UI completely functional/somewhat-polished.

[unfa]

About multi-display operation:

@fundamental you mean about a year to replace ZynAddSubFX UI? And then you could start working on LMMS?

[tresf]

@fundamental you mean about a year to replace ZynAddSubFX UI? And then you could start working on LMMS?

He was talking about the readiness of his GUI framework for Zyn which will make a good portion of the mockup possible. He never said he'd integrate it into LMMS, but rather was speaking broadly on behalf of its port-ability to non-Zyn uses (which I specifically asked about) :)

I'm curious how the messaging described as 99% OSC message driven with some sort of caching backend can or cannot be leveraged in LMMS. We use signals and slots for quite a bit of our stuff, but Vesa has expressed concern with it for real-time safety operations. Not sure if this message caching mechanism would help us in that regard or if it is more of a MIDI-type-channel that you are describing (or perhaps room for all the above).

-Tres

[rubiefawn]

Very nice. A complete UI overhaul probably won't be any time soon, but I do prefer single-window functionality and would be very pleased if this was implemented. Good work.

[fundamental]

On 03-31, Tres Finocchiaro wrote:

I'm curious how the messaging described as 99% OSC message driven with some sort of caching backend. can or cannot be leveraged. We use signals and slots for quite a bit of our stuff, but Vesa has expressed concern with it for real-time safety operations. Not sure if this message caching mechanism would help us in that regard...

Well, QT pretty much anything is very much unsafe to use anywhere near the realtime side of things. I've been procrastinating making a meta bug on the issue of realtime safety and how to make it an achievable goal within lmms. https://github.com/fundamental/stoat does a fine job at pointing out errors as they occur, but it's virtually impossible to fix any of these issues without some proper documentation detailing the overall architecture and assumptions of the components of lmms. Approximately what I think is essential to this is some sort of architecture level documentation (e.g. something akin to http://fundamental-code.com/tmp/architecture.html )

Some sort of message passing is the norm for effectively creating a realtime safe UI, via LV2 atoms, OSC, parameter ID pairs, strings, etc. Since the UI would still be qt based, using it for signals and slots would be possible, but I don't think it's a good idea for the long term plans of this project.

As per why I care about caching, it's mainly to avoid waiting for parameters when changing views which can be noteworthy in some of the cases with the current fltk UI.

--Mark

[Wallacoloo]

Is it possible to easily undecorate QWindows (i.e. remove each window's menu bar) and then have some layout engine that positions them in code? This would allow for an easy way to toggle between a tiled interface and the current windowing interface for multi-screen users. Should also reduce the amount of code affected.

A quick search shows that Qt may support this pretty easily.

[lukas-w]

@unfa

Seriously, I'm gonna put out some cash for implementing this.

Ever heard of bountysource.com? It's made for exactly that purpose :+1: https://www.bountysource.com/issues/10253686-new-ui-concept-single-window

[Umcaruje]

Ever heard of bountysource.com? It's made for exactly that purpose :+1: https://www.bountysource.com/issues/10253686-new-ui-concept-single-window

Oh yeah, bountysource is awesome!

Should we consider putting bounties on some issues with our donation money? That could lead to a faster development of some features.

[dednikko]

I'm supportive of that. It allows.non-devs like me to contribute more than ideas.

On Sun, Apr 5, 2015, 1:45 PM Umcaruje notifications@github.com wrote:

Ever heard of bountysource.com? It's made for exactly that purpose [image: :+1:] https://www.bountysource.com/issues/10253686-new-ui-concept-single-window

Oh yeah, bountysource is awesome!

Should we consider putting bounties on some issues with our donation money? That could lead to a faster development of some features.

— Reply to this email directly or view it on GitHub https://github.com/LMMS/lmms/issues/1911#issuecomment-89831381.

[tresf]

Our donation money does well to break even with expenses. Please don't misdirect funds to our donation account, leave them for the bounties. Bounties are a much more sensible approach to these bugs (any developer can step in). We lack the organization to make promises with our own funds. We're not prepared to dispurse and organize a large amount of funds, nor are we prepared turn our volunteers into paid devs. Separate bounties can attract new help without putting unnecessary burden on a relatively inexpensive and simple to maintain team. :)

So please pledge against bounties you believe in supporting, but don't think the current team can necessarily allocate more time. These bounties would put a price on a feature that anyone can fulfill (like new help). That idea sounds promising. :)

[curlymorphic]

oh, the evils of money....... :( :(

[Iniquitatis]

Sorry for bumping this thread, but this UI looks really amazing! Absolutely can't wait for the release. :)

[budislav]

Thank you all :)

@Lukas-W Why not Kickstarter?

[tresf]

@budislav that bounty page seems to have some good integration with issue trackers, and the bounty is on the bug itself so it puts onus (and thus contract) on the bounty hunter.

Kickstarter campaigns often are promoted, led and organized by the project team. Kickstarters also tend to promise much more features and have the tendency to raise monetary amounts so large that the project may not be able to handle with its given resources.

We aren't opposed to a kickstarter, but I argue we should be very cautious what we promise if we do. We need to be conservative on the amount of money we accept and the amount of features we promise. :)

[budislav]

@tresf I believe that bounty is the best for software project, but Kickstarter is better for bringing more money. If we make a detailed project plan and with posting on social networks this can work, like Krita https://www.kickstarter.com/projects/krita/krita-open-source-digital-painting-accelerate-deve/description

[Wallacoloo]

Ok, so is this thread about taking the UI code fundamental added to Zynaddsubfx and integrating that into LMMS? Or is the goal to just achieve a look similar to the mockup using whatever tools fit best with the current LMMS codebase?

[tresf]

@budislav too much money is a pretty big problem right now (I'd rather not go into specifics in this thread).

@Wallacoloo from what I am reading, a bit of both, depending on approach

[tresf]

@Wallacoloo to clarify a bit...

Single window is a pretty big initiative on its own and many of the tasks are quite separate from the theming initiatives.

To do this proper, we probably would be best splitting this initiative into some higher level components and go from there.

What is tempting about the Zyn GUI framework is that it may solve some of our problems and reduce efforts in the long run, but they aren't necessarily mutually inclusive.

Also, as you well know, this is a tremendous amount of work and requires clear vision and coordinated agreed upon effort.

For starters, I think we can start prototyping some single-window code enhancements and proceed from there. Once we have a clear vision, someone would have to help manage this effort and also help decide the effectiveness of leveraging the Zyn GUI code over rolling our own.

Said person would need a pretty deep understanding of the codebase (both DSP as well as RT-safe GUI).

Of course... Some of this will creep into (or replace) the 2.0 future map, so we'll need to be mindful of that as well.

[budislav]

Are anyone works on this??

[Wallacoloo]

@budislav Not to my knowledge (but maybe someone will pipe in differently).

LMMS is in "feature freeze" as we prepare for the next release, meaning that most of the developers are focusing on fixing bugs over adding major new functionality.

After feature-freeze though, I think we'll see some gradual progression along this front. I certainly plan to do my part, though I will probably be doing more "behind-the-scenes" work (My current plans involve cleaning up the code around how the UI interacts with the core - decoupling the two & hopefully implementing some sort of message passing scheme as fundamental has discussed in this thread).

[budislav]

@Wallacoloo thanks for answer. That sounds nice. Do you know approximately how long it will be "feature freeze"?

[tresf]

Do you know approximately how long it will be "feature freeze"?

Probably a month or so. The major holdup is one of our plugins "ZynAddSubFX" which went through a small code upgrade and getting it working without bugs has taken a considerable amount of time and attention.

[budislav]

@tresf thank you. I wish you success in that.

[Wallacoloo]

Btw, @tresf I think that after this release we should discuss the possibility of entering feature freeze at the same time the first (alpha) release candidate is released, and no earlier.

Spending 2-3 months in a feature freeze seems suboptimal, and it's a bit misleading when we're actually still waiting for one more feature to be completed (zyn upgrade).

[tresf]

possibility of entering feature freeze at the same time the first (alpha) release candidate is released, and no earlier.

Yeah, IIRC, the title "feature freeze" was introduced by Vesa in one of the few posts he has made since April. This thread probably isn't the place to discuss this, but since we're on the topic I'll elaborate a bit...

At this point, we could easily freeze and release RC1 without Zyn and @curlymorphic and I are both OK with this.

@Umcaruje, yourself (@Wallacoloo), @curlymorphic have made some progress getting Zyn functional, but we're still stuck on a few hard-crash items. Overall, Zyn 2.5 has postponed the 1.2 release since at least April. That combined with some very busy schedules and we're here in July. :)

As far as the feature freeze, that's normally done on a dedicated branch to allow master to continue getting features, but we've held off on splitting to a stable-1.2 branch intentionally so that the rather complex Zyn 2.5 code can be merged before the branch occurs, but we can certainly hold off on that until 1.3 (or 2.0, depending on who you ask).

Again, this is probably the wrong area to discuss this, but if we can reach a mutual decision on Zyn 2.5, we'll know how to proceed. Thoughts welcome.

-Tres

copied this post to 1991 - ZynAddSufFx Upgrade -- Curlymorphic

[tresf]

To bring this back on topic regards to the feasibility of the single-window interface, was the plan to move to the future Zyn GUI framework?

[Wallacoloo]

@tresf I don't think there's a solid plan for it. There are many ways to approach it though, and each with various timeframes, etc.

Doing things properly, we need to work on decoupling the core from the gui. If we go with a message-passing system, in which the gui thread/code never directly modifies anything but just requests the core to do something, then we can move away from all the locking and real-time issues. If we make this message passing system externally interfaceable (e.g. sockets instead of just a thread-safe queue), then LMMS becomes easily scriptable/automatable, and we can also write testsuites a bit easier.

Choosing a message-passing system based on that, it seems that OSC will serve us. I haven't read up too much on the alternatives, but if using OSC will make it easier to use Zyn code, it seems a reasonable choice. At first glance it seems to be an all-or-nothing type of thing: ditch Qt altogether and use Zyn's UI framework, or stick with Qt and avoid Zyn's framework. However, using a message-passing system gives us the power to have multiple different GUI implementations, so we can adapt the current GUI to work with the message-passing system and then slowly build up this new GUI to the point where it's ready to replace the existing one (it's still an all-or-nothing thing, but it means the changes can happen gradually and sustainably without ever leaving LMMS unusable).

My recommendation for proceeding with this is as follows: 1. Implement the requirements for the core to listen for OSC messages and queue them to where they can be processed at a safe time. 2. Implement the requirements for the gui to send OSC messages. 3. Go through gui callbacks individually and move the top-level callback code into the core, replacing the gui side with an OSC message. During this step, the core and gui are still in the same executable/process, and because this can be broken down into smaller tasks (i.e. commits that change just a single callback can be committed to a stable branch), it's much more manageable. We could potentially even do this directly in master to avoid merge conflicts, or we could maintain a separate branch & be prompt with pulling changes from master (I suggest the latter because it would be bad if we ever ended up with a permanent mix of slots and OSC code). 4. Once all OSC messages are implemented, work to move any mixed gui/core code into separate files and compile core and gui separately. 5. Add a new folder to master for the alternative UI and we can begin developing it, hopefully basing it off Zyn.

Open to discussion! Going the Zyn framework route, the GUI would have to be a ground-up rewrite. Having everything abstracted via OSC messages would make the rewrite much easier, though, as would being able to borrow routines from Zyn.

[tresf]

Having everything abstracted via OSC messages would make the rewrite much easier, though, as would being able to borrow routines from Zyn.

I find the OSC zyn-ui GUI rendering/framework portions to be particularly interesting because as Mark has stated, some of the C++ and/or QT code gets quite lengthy (or simply lacks functionality) for some fairly trivial UI concepts.

I feel like -- currently -- an overwhelming amount of our GUI code is statically managing layout because of the lack of a proper flow container for widget items and a decision to use statically sized windows and an MDI layout early-on in the project. Perhaps the initiative will vastly shrink the codebase as well as the learning curve to add plugin items. :+1:

[curlymorphic]

Most of my thoughts on this have been brought up in the last few posts, so I wont repeat them.

The use of a messaging system, and separating core form the ui would be a very good step in the right direction, and could be done as a separate project, With the size and complexity of this task I feel It may be a good idea to try and implement this, before any actual gui changes.

As for messaging systems and RT safety, I have no real experience in this, I have done reading on the subject, Seeing as we are already going to be including OSC libs , both lo and rtosc, with the ZASF 2.5 upgrade I feel we should consider utilizing them.

@fundamental linked me his tutorial on using rtosc, but as of yet I have not had time for that sub project. but is something I would like to spend the time on, before undertaking this task on a codebase as large as lmms.

https://github.com/fundamental/rtosc-tutorial

[musikBear]

a micro tiny comment on the freeze lmms is -one- method away from having the feature, that perhaps has been the most requested over time. Lmms 1.2 can have ghost-notes, if only 2155 was addressed..

[tresf]

@musikBear please don't lobby for other bug reports in this one. :+1: We all know the popularity of ghost notes. :+1: Popularity doesn't drive features (yet). Perhaps we'll be there some day, but for now, we do all what we can.

[fundamental]

@Wallacoloo yeah, that sort of multi-stage implementation/upgrade plan is basically what's required. Fixing up architectural problems is a messy affair, but doable when the rework is an organized effort.

[softrabbit]

I'm not out to pee on anyones parade, but after letting this proposition sink in for a while, I can't help but wonder. And I've seen enough designers being very firm about their Vision, so I just have to ask:

Looking at the blue drawing of the window subdivision, I read things like "MIXER or AUTOMATION EDITOR or MIDI EDITOR". Is the intention really to:

• Force users to have e.g. the mixer in the middle and the instruments at the bottom, or will the layout be configurable at least to the degree that I can move these top level blocks? I've always had the mixer at the bottom and I'd really like the freedom to put it there in the future.
• Make certain parts compete for space? What if I like to have both mixer and automation editor open and want to allocate a lot of space for them? And why have the BB-editor share a block with the song editor? I'd understand better if it was BB &/or MIDI editor, as those are kinda parallel concepts.