Params + Mouse Interaction

[robbielyman]

This is a tracking issue for adding mouse interaction to the params menu and improvements to mouse handling overall.

• [x] Zig: capture scroll wheel (or trackpad scroll, which is hopefully the same) in SDL
• [ ] Lua: "popup" menu (or should this be an OS thing? I'd rather have it be internal—but only for aesthetic reasons)
• [ ] Lua: widgets?
• [ ] Lua: pages vs scrolling lists?
• [ ] Lua: more highlights for state (currently clicked, for instance)
• [ ] Lua: other conveniences that could be broken out for scripting reuse?

[robbielyman]

@andr-ew @dndrks + anyone popping by, feel free to chime in with comments or PRs to the linked branch!

[andr-ew]

eee ty ~ I'll link my first draft mockup in case anyone else is tuning in

https://www.figma.com/proto/xMtv89DyWSi0jtMScDbZcU/seasmstress%2Fparams?type=design&node-id=1-10&scaling=min-zoom&page-id=0%3A1

i think, to summarize (and correct me if I'm wrong), we liked the hover states, but thought it was a bit too cluttered with all the indicators, and generally felt there were more opportunities to make things more interactive/intuitive if we strayed further from the norns paradigm

[robbielyman]

scroll wheel = trackpad, thankfully. I added screen.scroll() (for users) and paramsMenu.scroll() functions (for us) in Zig.

In Lua, it honestly might almost make sense to have a hidden control param for scroll x/y position—the reason being because SDL outputs a delta for scroll, naturally.

[andr-ew]

thxxxxxx @ryleelyman !

---

on the design front, I updated the prototype above to drop the single triangle for controls – I think it's the right look for that interpretation

and then, I tried something completely different, based on @dndrks observations:

some notes

• a breaking change from norns is that most everything, particularly param.names, should have character limits – I'm not excited about coding a lot of crazy text wrapping rules to accommodate any text. but I'm honestly really into this limitation because the group names & separators are there to add context
• not attached to the specific look & feel here (though I do want it to look good). the main thing I wanted to demonstrate was a tiling approach to the list, which, I think, is always going to be the better use of space when using widgets. and sticking with the script-assignable single color cause I think that's 💯.
• to start off, I'm trying to use as few widgets as possible (just three, which actually lines right up with the hidden use of controls under the hood)
• i should mention, it is a bit more ambitions, but I am up to coding all of this or something similar, just might take me a bit of time. actually I'd sort of rather start from scratch than build on top of stuff copied over from norns, just how my brain brains
• the (?) tab is for listing keyboard shortcuts – if we're doing this much for the param display, I think it would be wise to keep it at that & do most of everything else (midi mapping, save, load, etc) via keyboard shortcuts, which would generally have a nice feel to it as well & keep things accessible.
• for text size accessibility, I think we can keep it simple & add some keyboard shortcuts to just scale the whole window up & down, fonts & all.
• I'd be down for light / dark themes (probably user (rather than script) configurable, via kb shortcut).

would love to hear thoughts ! I might come back & see what focus or hover states might look like, if any, but this pretty much gets the core idea across I think

[robbielyman]

Wow, thank you for this! it looks great!

I ... sigh, I'm interested to hear what others think, but I honestly probably want to stick to 04b03 and seamstress's screen API for the graphics. I'm aware that SDL / a laptop in general / a proper gui creator library is muuuuch more capable than the use I'm making of it, but I think the lofi look is charming and keeps expectations appropriately low. I guess I'm keenly aware that this hurts accessibility for users of, e.g. screen readers and don't really have a solution there...

(I guess that because 04b03 is not monospace there'd be a small amount of asking it "ok how many m's can fit in X pixels", but it's easy enough to copy norns's "trim string to length" function.)

[andr-ew]

I honestly probably want to stick to 04b03 and seamstress's screen API for the graphics.

oh to clarify I do agree that this should be written entirely in the lua layer ! this was designed with that in mind, but I made the mistake of not reviewing seamstress's screen API vs norns or checking whether there would be an ask for added functions. it looks like a couple functions are missing when it comes to this particular look - namely rounded corners & font size/font-face options. but it looks like text, rect_*, line & arc can cover all the rest w/o modification. but we don't absolutely need rounded corners & the font size/face could continue to be homogenized across the app if that's a preference.

I could try an alternate w/ 04b03 if you'd like to see what that looks like. I wanted to try out a live.dial-style widget & as a designer I'm less excited about mixing circles with a square-y pixel font-face, but we could go in a more square-forward direction, probably with faders rather than dials :) I like something about how dials feel when using a mouse, that's why I went with a round-friendly approach (and roboto mono still felt like a teeny nod to the community, if a departure from norns & the low-res vibes).

[dndrks]

woahhhh @andr-ew this is such a gorgeous mock-up!! what an energizing image !

i'm also curious how a lower-res approach would feel -- the norns UI library feels like it has all the right stuff, especially slider's options for specifying vert, horiz, direction, and marker positions.

that mock-up absolutely nails the pages idea, though, which feels like a super-actionable first step in terms of additions to the codebase. i can take a swing at building out the data structure, if that'd be helpful?

just pasting the pages idea from our DM's here

> i was actually considering what things might look like to build ‘pages’ of controls, rather than always forcing the scrolling list. this might help break things up visually for scripts with lots of parameters. eg. what if each of those separators in your example could be its own page? then it could be easy to quickly navigate from ‘clock’ to ‘filter’ controls, without disrupting your place on either page. this breaks from norns, but perhaps usefully?

re:

I guess I'm keenly aware that this hurts accessibility for users of, e.g. screen readers and don't really have a solution there...

i'm curious if, as the project continues, if someone with the know-how might be able to help with this -- not necessarily to make the params window screen reader friendly, per say, but who could help us understand how to hook into the brilliance of the original params code + structure to build out alt navigation of params. i need to keep reminding myself that accessibility isn't always simply 'the visual UI of an application can be navigated by a non-sighted artist', but is more often 'a non-sighted artist can use the application as fruitfully as a sighted artist'.

[andr-ew]

i'm also curious how a lower-res approach would feel -- the norns UI library feels like it has all the right stuff, especially slider's options for specifying vert, horiz, direction, and marker positions.

bring back the chonk !! I might take a brief hiatus from these mockups but I'll definitely check back in with a lowres + sliders look to the pages/titling layout. I'm glad the actual layout/ux of it is sticking w/ ya'll though : )

that mock-up absolutely nails the pages idea, though, which feels like a super-actionable first step in terms of additions to the codebase. i can take a swing at building out the data structure, if that'd be helpful?

ooo I'm curious what you have in mind here! like an alternate form of add_group ? I actually imaged the pages/tabs mapping pretty transparently to group and those little channel 1/channel 2 headers mapping to separator – but I do find that add_group is a little clunky to use on norns, and would probably be even more clunky in this context

[robbielyman]

I was just thinking it would be great if params:visible or some other mechanism existed to say whether a given param is currently being displayed—would allow something like the clock:tempo param to redraw the params menu only when you're looking at the clock page...

[robbielyman]

I'm still interested in mousable param interaction, but I think maybe starting smaller seems more realistic. I'll close this out for now.