Expand functionality, add more visualization options

[djfun]

As introduced in https://github.com/djfun/audio-visualizer-python/issues/9#issuecomment-303070367, the idea is to have the ability to add more visualization options. I made a UI mockup with http://pencil.evolus.vn/ how such a system could look like: If someone wants to extend this mockup, I can upload the .epgz file.

I would like to have some thoughts, opinions, new ideas on this suggestion and possibly already people who would offer to help with the programming.

[rikai]

Seems like a great start with decent potential for growth to me, but i did also just wake up, so i'll think on it some more. ;)

[djfun]

ok, cool! thanks btw. for featuring this project in the podcast

[rpodcast]

While this comment is more about the backend and other visualization options, I was having discussions with @rikai on another channel about additional visualization ideas. He found a powerful music visualization suite called projectM which mimics the milkdrop visualization tool used in the classic WinAmp player of yesteryear. I've begun a dialog with their team on how this framework could be invoked and general guidance on creating wrappers to it (such as a python wrapper), so that eventually it could be called from this visualizer. This is all just a suggestion, and if there is a way you could accomplish a similar thing in this audio visualizer directly then that would likely be a more logical path forward.

[djfun]

I think the visualizations of projectM could be a great component/plugin in this new extended tool. The user could add this projectM component and then for example add a text over it. The current visualization in audio-visualizer-python, the moving bars, is created frame by frame as PIL images and then send via pipe to ffmpeg. So I think a python wrapper for projectM (or any other visualization tool we would want to have a plugin for) would need to be able to either send the visualization data as single frames, then we could do the compositing with other components like text in python, or it could also send it as full video, which we could then either read and composite in python or let ffmpeg do the compositing. If you have any more questions regarding the technical side of this tool, feel free to ask. We could even set up a telegram,threema,irc,.. group chat if that makes it easier to discuss/coordinate.

[rikai]

@thercast We did discuss this a bit in #9 already... Probably should create some kind of tracking issue for the idea.

@djfun I've personally got telegram and discord open all the time on my desktop and phone, so either of those would work fine for me, though i'm more of a 'concepts' guy that a real coder. 👍

[tassaron]

I like that mock-up. I notice it says the preview is "possibly interactive", which is interesting. I was thinking it would be nice to be able to specify the title text position by clicking the preview itself, instead of typing arcane numbers into text fields. Is that something like what you meant?

This UI looks way better than the current one... My only thought is on the placement of the progress bar. You could have the progress bar at the bottom of the window by itself, with a large 'Create Video' button above it in the left pane under the Mandatory Options. The left pane should be smaller than the right pane so there's more room for a large button there. I have no idea if this would actually look better, it's just a thought.

I would love to help with the programming side. However I don't know if I'm a good enough programmer to make a plugin system... I could try but it sounds hard :) I use telegram and irc.

[djfun]

Yes, I was thinking of being able to drag the active element in the preview window to reposition them and possibly changing their size.

When I made this mockup, I was not entirely satisfied with the positioning of the progress bar either. Maybe we want to move the select music etc. buttons over where there is still blank space under the add/remove component buttons. Then the space for options of components could be the entire width of the window. What I don't want is, if it is avoidable, that you need to have the program in full screen on a laptop monitor. The preview should be able to grow, if the window size gets bigger.

[djfun]

@tassaron2 @thercast There is now a telegram group at https://t.me/joinchat/AAAAAA2PGPf1Us18F49_ag to coordinate the project and exchange ideas

[tassaron]

Hm. I think I like the mock-up better because it splits the window into mandatory settings on one side and component-related things on the other side. It seems nicely organized in that way. I suppose if you put those buttons with the components, then you could shrink the window significantly by hiding the component options since they would be on their own row... But then certain 'mandatory' features of components would be hidden, like the title text.

So a possible design could be... preview window in top-left pane; all mandatory options in the top-right pane, starting with the Main options (select music, background etc.), then an area for adding/removing components, then the mandatory options of components organized by subheading; and the bottom pane is a big area for optional options (heh), again organized by subheading. The optional area could then have the Main optional options (frame size, etc) followed by the component options in order.

This way all the extra settings could be hidden away behind a button if so desired, and the window would be a lot smaller until someone needs to change the frame size or colours... But that might be coming at the cost of being a more confusing design since it splits the settings up. I don't know.

Components need to have a standard definition file giving them a name, a path to plugin code or something, and define what the mandatory and optional settings are... And any setting could have a default state defined for if it, which would be added to the form in audio-visualizer when it's constructed. I don't know what the best method for actually communicating with the plugin/component code would be, though. The components could simply be Python modules instead of requiring a definition file. Hmm.

I would also like to implement the ability to drag and drop files into the fields, so you can use a file manager to select files. I've tried adding that already and it seems easy.

[tassaron]

I made a mockup of what I said above, but this makes the window quite long. I'm not too sure about that. However it shows what I meant by having one section of the window hideable underneath the horizontal separator, with the mandatory settings separated from the optional settings. (The only mandatory setting here would be the title text.)

Instead of a bunch of little boxes, the settings could be in a tabbed interface. It may look tidier that way.

[tassaron]

Technically the components add/remove area could itself be considered an advanced setting and hidden, if the button is moved to the left. The grey separator marks where the window could be shrunk to. This would make the window less wide.

[dv-anomaly]

This is my take on the design, based alot on the submission by @djfun. If we need a more settings button like some o the other mockups here, there is plenty of room in the top right section to do so.

[dv-anomaly]

Slight design change. A preset would be the saved state / configuration of a component. A project would be your entire configuration, including your components list, component configurations, output settings, etc.

UPDATE This UI is currently being implemented in this branch. https://github.com/IamDH4/audio-visualizer-python/tree/feature-newgui

[dv-anomaly]

Current iteration of the new UI has been pushed to the feature-newgui branch. Several items/features are still a work in progress. Need to change the layout for project and preset buttons. Some backend code to implement. We are looking for feedback on the design.

Thanks!

[dv-anomaly]

Please dedicate this thread to discussions about the UI only, projectM has it's own thread at #22. If you have suggestions for other visualization libraries please create a new issue.

[dv-anomaly]

This is the latest iteration on the design. Many new features implemented. I am considering this issue closed. Please report bugs, feedback, and feature requests by creating a new issue. We are currently on a feature freeze until we hit our first milestone.