LMMS redesign concept

[budislav]

(Click to enlarge)

---

Hi all, My previous work for Zynaddsubfx has same purpose to dock all windows in one (link is dead) http://budislavtvp.deviantart.com/art/ZynAddSubFX-UI-Concept-2014-455890191

It is in development phase > https://github.com/fundamental/zyn-ui-two

Link to full concept: http://budislavtvp.deviantart.com/art/LMMS-UI-UX-Design-Concept-Single-Window-523696539 https://www.behance.net/gallery/38503021/LMMS-UIUX-Design-Concept-Single-Window-Interface https://www.behance.net/gallery/194917259/LMMS-Redesign-Concept

Full version (mirror): full version

[J5lx]

IMHO realizing that drawing similar to the way Blender's UI works (see edit) would be a really nice solution to both problems you listed. However I believe Blender uses some tightly coupled Python code for it's UI, so I don't now whether it could just be dropped into LMMS or had to be written from scratch.

EDIT: Just to be clear: I'm mainly referring to the way Blender allows for rearranging the window "areas" (editors), not necessarily to the other stuff.

[budislav]

Ok, I made solution for configurable layout and multiple monitors.

User can click on tab then drag and drop on some other section. He can save layout from menu bar and he can switch between them from drop down select button from main toolbar. LMMS should have some default layouts like mixing, arranging,.. User also can collapse tabs by double click on tab. Left and right sections will collapse in vertically line. There is only one limitation. Instrument and effect rack have fixed height so this rack can't dock on left or right, just on top, middle or bottom sections. Controller rack have fixed width so it can dock only on left section.

I think this system would allow enough freedom to set a wanted layout. There is too many combination to show, so I would not draw every of them.

[TheColorRed]

If anyone has ever used Unity3d, the editor looks very much the same with tabs and stuff like that, and the core code is separated from the GUI code and connected with an API of sorts, where the GUI is mainly written in C# and the core is mainly written in C/C++. Users of the software can also modify GUI when creating plugins for the engine; they can add things such as custom windows, menu bar items, etc. all using C#. I started a mesh editor that would allow you to morph the mesh of 3d objects. All because of this C# API that they have. Also you can edit the C# code, save it and the editor will compile it and update the GUI all without restarting the software.

This could be helpful for those who don't want to edit core software code to be allowed to add some cool features to the software, such as a different way to edit beats or a different way to use the piano roll. The options would be endless. Using a higher level language such as C# will allow for more people to be able to add these features/plugins to the software, since the language is easier than C/C++ for most people.

BTW @budislav this looks awesome! I love using software that looks pretty :) For game development I use Unity3d because it is extendable and looks great! Software that has a UI that looks like it was made in the 90's is very unappealing.

[curlymorphic]

@TheColorRed

I have used Unity3D, and agree the ability of being able to code addins in c# is very nice, and in that scenario works well. I also currently use AutoDesk Revit that has the same feature, but from my searches is not used as much. I feel this works because the majority of users of unity are already coding in c#, most revit users are not coders.

This could be a massive effort, on the part of lmms, To include a scripting language, and write an api for this use.

This is a valid idea, but I am unsure if lmms has the resources to implement.

[J5lx]

This could be a massive effort, on the part of lmms, To include a scripting language, and write an api for this use.

Yep. I had to embed Mono specifically once and it was quite a challenge since there’s no documentation except for the most basic stuff and the API docs, and even those are not quite comprehensive. Many functions are not even documented at all IIRC. At the end I found an alternative, so I switched happily.

If a scripting language is desired, I believe Python could be a better choice - I didn’t try it myself yet, but when I was working with Mono I read many times that it would be rather easy to embed.

[budislav]

@TheColorRed Thanks! I always put UI on the first place :)

[budislav]

Can somebody add UX label on this topic?

[Umcaruje]

@budislav done.

[budislav]

@Umcaruje thanks :)

[M374LX]

Nice UI concept, but will the user be able to view multiple instruments at the same time?

[8tab]

Why do you want to view multiple instruments at the same time?

On Thu, Aug 6, 2015 at 3:17 PM, Alexandre Almeida notifications@github.com wrote:

Nice UI concept, but will the user be able to view multiple instruments at the same time?

— Reply to this email directly or view it on GitHub https://github.com/LMMS/lmms/issues/1911#issuecomment-128359351.

[budislav]

@M374LX no, because there is no need for that. User have all effects in one rack with instrument and he can quickly switch between all instruments and all type of effects in one project.

[mamarilmanson]

:+1:

[michaelgregorius]

@budislav Don't be too quick with saying that there is no need for being able to look at multiple instruments. I can imagine that LMMS has many different users with many different needs.

Also please keep in mind that LMMS supports VSTs which provide their own GUIs with different sizes anyway. So if you want to present internal plugin GUIs in a fixed place VST plugins will have to be presented in a different manner which I can imagine might be confusing for users.

I also think that the current fixed size of the existing internal LMMS plugins puts a constraint on the features that you can implement for them because there is not too much space to put the controls on to compared to some VSTs. I think that different sizes and layouts give quite a lot of character to VSTs and I think it would be great if LMMS would provide a similar flexibility.

[tresf]

To @budislav's point though, this isn't a deal breaker and his mockup is a concept.

I find the multiple-instrument interface to be hideous, counter intuitive and reminds me of a deck of cards. The only reason I use it is because of what I feel lack of functionality (not a complaint, an observation) as it allows me to do a side-by-side comparison of instrument knobs which should probably just be bulk "copy/pastable" from a UI perspective (not to mention that many of our plugins don't offer a visual spectrum feedback, so the knob positions can be arbitrary from a visual perspective)

But multiple instruments really comes down to "context" and in the context of a track, you generally are tweaking a single instrument. If there is enough demand by our users for multiple instruments interface, we find a way to make it possible.

But design is intended to be driven by usability, but in the end it is largely driven by opinion. Since opinions never align, we'll never agree. From what I'm reading above, the overwhelming feedback is to move to this design, so if we all work toward this single-window interface, we have to make some difficult compromises and the fastest way to that is to 1. Drop the multi-instument interface or 2. Brainstorm and provide the work-around which makes both wishes possible.

-Tres

[budislav]

"you generally are tweaking a single instrument" Thank you @tresf . This problem can be solved very easy by adding option for splitting on two rack horizontally or vertically so if user have two screens, vertical split should be better choice.

[M374LX]

"you generally are tweaking a single instrument" Thank you @tresf . This problem can be solved very easy by adding option for splitting on two rack horizontally or vertically so if user have two screens, vertical split should be better choice.

Great, @budislav!

[musikBear]

looks amazing! :+1: :100:

[eagles051387]

After reading through this thread, this is just a suggestion Why not give users the ability to choose between multiple windows like we have no or single window as per the mockup?

On Sat, Aug 8, 2015 at 2:29 PM, musikBear notifications@github.com wrote:

looks amazing! [image: :+1:] [image: :100:]

— Reply to this email directly or view it on GitHub https://github.com/LMMS/lmms/issues/1911#issuecomment-128973267.

Jonathan Aquilina

[musikBear]

@eagles051387 Has a point there. Also How is this single window-concept looking on older screens? Will the single-screen design only work well on wide-screen? There are still many users with older monitors, how will those see the design?

[eagles051387]

How are layouts handled right now? Can they be done through css?

On Sun, Aug 9, 2015 at 12:09 PM, musikBear notifications@github.com wrote:

@eagles051387 https://github.com/eagles051387 Has a point there. Also How is this single window-concept looking on older screens? Will the single-screen design only work well on wide-screen? There are still many users with older monitors, how will those see the design?

— Reply to this email directly or view it on GitHub https://github.com/LMMS/lmms/issues/1911#issuecomment-129150602.

Jonathan Aquilina

[Umcaruje]

How are layouts handled right now? Can they be done through css?

Sure, you just need to add

* {
    single-window: true;
}

To your style.css.

If this doesn't work it probably can't be done through CSS.

[eagles051387]

If styling is done through CSS I am more then willing to work and implement bootstrap based styling that way we can work on any screen.

On Sun, Aug 9, 2015 at 2:03 PM, Umcaruje notifications@github.com wrote:

How are layouts handled right now? Can they be done through css?

Sure, you just need to add

• { single-window: true; }

To your style.css.

If this doesn't work it probably can't be done through CSS.

— Reply to this email directly or view it on GitHub https://github.com/LMMS/lmms/issues/1911#issuecomment-129171176.

Jonathan Aquilina

[lukas-w]

Hehe

[tresf]

How are layouts handled right now? Can they be done through css?

You've asked this before. Find the last time you asked and the answer will be the same.

[budislav]

@musikBear "How is this single window-concept looking on older screens?" single window workflow is looking better on older screens.

"Will the single-screen design only work well on wide-screen?" No.

"There are still many users with older monitors, how will those see the design?" Perfectly. Are you one of them? Which resolution your monitor have?

This workflow will works better on smaller screens than present UI. I worked on this over 5 month and I brought it to perfection. Everyone who want to say what is wrong or what is not maybe he just need to study UX and UI before.

This can help http://www.jiscdigitalmedia.ac.uk/guide/graphical-user-interface-design-developing-usable-and-accessible-collection,

http://www.usability.gov/what-and-why/visual-design.html

This are statistics about screens resolutions, the latest data from January 2015 are the most important http://www.w3schools.com/browsers/browsers_display.asp

[musikBear]

This workflow will works better on smaller screens than present UI.

That is all i needed to hear Very fine work indeed.

[mamarilmanson]

any updates about this UI? :D

[budislav]

I would like to know :)

[tresf]

any updates about this UI? :D

Minor, yes. @Wallacoloo has begun implementing the first stages of the OSC messaging system ("with the goal of reducing the coupling between the core & gui")

Choosing a message-passing system based on that, it seems that OSC will serve us. I haven't read up too much on the alternatives, but if using OSC will make it easier to use Zyn code, it seems a reasonable choice.

via https://github.com/LMMS/lmms/pull/2252.

But as he originally stated:

@tresf I don't think there's a solid plan for it. There are many ways to approach it though, and each with various timeframes, etc.

So, slow and steady. :+1:

[budislav]

Great news :+1:

[michaelgregorius]

I understand that the decoupling of the GUI and the core code is related to the current GUI. But I think the new GUI elements could be implemented independent of the work done on the decoupling anyway.

One way to do this might be to create a new separate folder for the new widgets and to start implementing them. These widgets should not be used in the "productive" code, i.e. they should not be connected to any existing core functionality yet. This would mean to only implement the rendering and perhaps some useful signals and slots which are not related to any core functionality. The intention is that due to the decoupling the core code will likely change too much anyway and that therefore we should only connect the new widgets once the core is stabilized.

We could also add a new project to the CMake file that's used to build a prototype of the new GUI or to test the new widgets.

Obviously this would still be a lot of work but it should be possible to start this in parallel (with the risk of the new widgets not being incorporated of course).

@budislav What font did you use in the concepts by the way? Is it a font for which it would be ok to put it into applications resources and to distribute it?

[budislav]

@michaelgregorius That font is Droid Sans. Yes it would be ok to put it into applications resources and to distribute it.

[tresf]

@michaelgregorius I agree the tasks aren't mutually exclusive, but remember that this new UI (AFAIK) is hypothetically QML based and uses a completely separate work-in-progress framework... https://github.com/fundamental/zyn-ui-two

[fundamental]

Yeah, the zyn replacement UI framework (tentatively named 'zest') is quasi-qml based. Don't judge the current progress by the lack of progress in that particular repo though. Right now I have a bunch of stuff done locally while I'm working out how OSC interacts with the reactive dataflow of qml and the layout stuff is getting rewritten yet again to upgrade it to a fully global constraint optimization setup. Everything is still in pretty early stages, but it's going to eventually get there.

If there's someone really interested in digging into it I can try to make an effort to get the commits pushed off of just my machine, but at the moment there's more design notes than a working toolkit.

[mcanthony]

@fundimental I have been (rather silently) monitoring the progress of your Zyn related efforts as well as the integration of @budislav work into LMMS and I have been especially interested in the architecture you have been describing here and as it relates to your ZynAddSubFX overhaul in particular. I intend to port Zyn to NaCl for the WAM project: https://github.com/webaudiomodules - in which I think a lot of your work will be crucial in my task of retrofitting an HTML5 based UI.

I am a C/JS programmer and I pretend to write C++ - if I can be of any assistance to your efforts I will surely do my best.

[mcanthony]

It appears I misspelled @fundamental in my last post (which is really special of me since github has auto-complete) so here's a message just for the purposes of @mentioning @fundamental.

[fundamental]

@mcanthony I should be able to get the WIP code up this weekend, though until a particular mruby bug is fixed, it's not going to be able to do too much. For your goal of porting zyn, I don't really see how my effort really changes things all that much, though if you wanted a standalone zyn with a browser based UI I could see how the OSC layer would come in handy.

As this toolkit seems to be of general interest beyond just it's use in zyn I've setup a mailing list for it: https://www.freelists.org/list/mruby-zest

[mcanthony]

The UI/OSC layer is most interesting to my project, as for porting it eases the pain of replacing the UI code with something that fits with the paradigm of CSS driven UI even if the code itself is not useful to me, the concepts behind the construction of the GUI framework for Zyn have gone a long way toward bettering my understanding of the issues involved, and seem to be far more fitting to be remapped to a browser context. Your ideas and statements about how The porting side of things, as far as signal processing shouldn't be very OS bound, so this should be fairly straight forward. Perhaps it was heavy handed to say it's crucial to my goal, it has been more crucial to me goal of understanding the process of making such modifications to the fabric of Zyn's UI arch such as you have been made to do.

Aside from that I am simply a fan of the UI and the synth in general, so I seek to support it's betterment. I may end up using the standalone Zyn library if the full version of the app itself becomes too problematic to port.

[fundamental]

@mcanthony well, that took longer than it should, but here's the current code: http://fundamental-code.com:8080/index

It's not moving quickly, but I think it will eventually make it to the finish line.

[Reaper10]

I thought of a GUI compromise

[jackokring]

Zyn as I understand uses the FTLK toolkit, it is very memory efficient, an original xfce desktop toolkit for a small X desktop. It does not use Qt in the settings window. It is an embedded X11 component in Qt.

Lest not forget, it's sound output that matters. For those who note such things FTLK (faster than light tool kit) is statically linked, and design options are definitely not GTK or Qt level. But then again it won't flood the processor memory bandwidth with excessive pixmaps. I'm not sure if the project is 2D accelerated, but a replace grey square with scaled pixmap could add much. But that would be a upstream on the FTLK project perhaps.

EDIT: Try the xfce4 or (higher?) desktop to see what's possible. EDIT: Using a Qt keyboard exported as an X11 control would maybe (although debatable due to Qt container code size) improve performance of multi-part polyphonic instruments using many tracks of such.

http://www.fltk.org/documentation.php/doc-1.1/toc.html and the T and L are backwards in my post.

EDIT: Possible fast to do optimizations:

1. Get rid of the keyboards in Zyn views, the one in the main instrument window is fine. (makes code smaller)
2. Change the FLTK default colours.
3. Change the default button style.

As an extra is a synth design I'm working on. It could do with some tart. The limits are, the knobs must have the same centres and diameters, and be equally spaced. (the last 3 on each row maybe a pixel or so out. And an icon 48*48 too.

[tresf]

faster than light tool kit

Fast, Light Toolkit

Lest not forget, it's sound output that matters [...] that would be a upstream on the FTLK project perhaps.

I'm not sure what you've inferred from this thread, but Zyn's changes are happening upstream and this thread assumes whatever Zyn uses, we can also use, not the other way around.

Get rid of the keyboards in Zyn views, the one in the main instrument window is fine. (makes code smaller)

What are you referring to, our instrument plugin keyboard or one of the mockups? It's hard to know what context these comments are made for/in.

As an extra [Jacobi] artwork is a synth

I feel that is more appropriate in #2452.

EDIT: Try the xfce4 or (higher?) desktop to see what's possible. EDIT: Using a Qt keyboard exported as an X11 control would maybe

Now I'm really confused.

I'd recommend you scan the fundamental posts from this thread, he's the one writing the GUI framework which may someday make the mockups in this thread possible. :+1:

[jackokring]

I'm not sure what you've inferred from this thread, but Zyn's changes are happening upstream and this thread assumes whatever Zyn uses, we can also use, not the other way around.

True but Zyd if it sticks with FLTK, would have it as a further upstream.

What are you referring to, our instrument plugin keyboard or one of the mockups? It's hard to know what context these comments are made for/in.

The keyboard in the basic which is not there in the advanced view.

I'd recommend you scan the fundamental posts from this thread, he's the one writing the GUI framework which may someday make the mockups in this thread possible.

Sounds good, but when shuffling pixmaps (intense GUI overlays and effects), the sound also travels along the memory bus in the system too. I like the mock ups, and think they are reasonably efficient on mathematical calculations needed, and pixmaps needing to be transferred to graphics memory for rendering. But me I'd recommend square pots to not need a sin/cos table occupying layer 1 cache in the processor. #2460

[tresf]

The keyboard in the basic which is not there in the advanced view.

Then please post on the Zyn project page, we aren't discussing that here.

I'd recommend square pots to not need a sin/cos table occupying layer 1 cache in the processor.

Recommend this in what exactly?

[fundamental]

@jackokring I have no idea where you're getting this concern about pixmaps from. There's no other mention about them in this thread and the toolkit which I'm writing (which is NOT FLTK) doesn't even have a widget which supports pixmaps so far.

Profiling performance is a key part of what I'm doing with this toolkit as I'm trying to have very reliable 60+Hz full screen updates, 20+Hz resizes, and 10+Hz full UI reload (as code hotloading is supported). Once partial redraws are added back to mruby-zest (from the zyn-ui-two code) a very trivial amount of CPU, bus, and GPU time should be spent for each frame rendered.

[jackokring]

Recommend this in what exactly?

In a fastest possible toolkit. A bit of square humour. The fastest render would be a texture map on a surface rotation in openGL.

I suppose the initial confusions/confisions on my part were about the application of this UI scheme to the whole LMMS or just for plugins.

[budislav]

All right, so I got a mail from Alexandru Celea film producer from Mad Horse films about using my lmms ui designs for their new short movie about a group of high school kids who start a music label.

They would love for those kids to have this UI on their screens when they are working, also new zynaddsubfx gui. I am not kidding! Of course I gave them permission to use them. :)

That forced me to redesign LMMS one more time. Modern, flat and clean.

[rubiefawn]

Please make this gorgeous ui a reality

[IvanMaldonado]

Congratulations! I feel very excited for you, I have been inspired by this work of yours on my humble themes, I really would like to see your work in action some day.

Enjoy the success you just got, because you deserve it!