[Discussion] How can we bring more of a middle-ground of functionality to Hammerspoon

[cmsj]

Quoting @asmagill from #2373

Where I'm going with this is that I've had a chance to review CommandPost's https://github.com/CommandPost/CommandPost/blob/develop/src/extensions/cp/ui/axutils.lua and some of these look really useful and especially helpful for someone new to accessibility objects and how they're usually organized within an application. We don't have a really good place for helper libraries like this... modules for fundamentals, Spoons for "completed" works, but nothing really in between that's entirely lua code, but not complete enough by itself to be a Spoon.

To get this module moving, I'm going to work on integrating some of these tonight into the module, but I thought I'd put my thoughts out there... do we need something in between modules and Spoons for "helper" libraries? From the developer perspective, I lean towards yes, but wonder if it might just add confusion for newer non-programmer users?

I agree very much with this. I think we've done a great job providing loads of low level blocks of functionality, and we have a decent-ish array of Spoons, but we should also consider the middle ground.

I don't have any concrete ideas about this yet, but I've been thinking about some guiding principles for how to address this:

1. Try not to muddle our already extensive API docs, with intermingling of low-level and higher-level functions
2. Steer users towards this middle-ground
3. Try to find middle-ground between the "no opinion" APIs and "highly opinionated" Spoons. This work should be larger blocks that provide commonly used functionality, but leave the manner of its use up to the user.
4. Encourage users to participate, in a similar way to how Spoons provide a common structure for others to leverage

The idea I tentatively have right now is to add a helper.lua to modules, which offers functions/objects in a hs.$MODULE.helper namespace. I see it addressing the above principles:

1. A separate namespace for each module clearly separates it in the API docs (in hindsight I might have chosen to invert this and have hs.$MODULE provide the higher level functionality, with hs.$MODULE.something providing the low level blocks, but we are where we are)
2. Rewrite the GSG as we go, to use Helpers, and prominently mention their existence in the top-level docs for each module.
3. I think this is going to come down to "we'll know it when we see it" kind of test
4. This one is probably going to be the hardest to achieve, but I have one idea - when we talk about writing configs, or in our GSG, we could explicitly encourage users to insert their utility code into the hs.$MODULE.helper namespace. This breaks down as the helpers span multiple modules though.

[randomeizer]

I'm not sure if it would be considered a "middle ground" API, but cp.app is basically a module built on top of a few other low-level modules from hs (axuielement/fs/etc). It wouldn't work without them, but can kind of be used without knowing three others exist (other than axuielement, which is exposed directly). But is it too complex for what you're talking about here?

[cmsj]

@randomeizer I suspect that cross-module helpers will pretty much destroy my idea as it currently stands, so we should go over some other possibilities, like a hs.helpers module, but then how do we subdivide that to stop it becoming an un-navigable mess of different things...

[randomeizer]

My example is a separate framework really, more than you were talking about. Keep it simple.

[asmagill]

Just so we're all on the same page, I have a couple of questions that we should probably iron out sooner rather than later...

• Will these helpers be a part of the Hammerspoon release, or will they be something the user has to add themselves, like the Spoons? If the later, will there be a loader built into Hammerspoon (like the SpoonInstall spoon -- I assume from reading about it's description, never actually used it myself; since I have the whole repo on my machine, I just symlink to the src dir)
• How will these be documented (beyond the mentioning in the API header sections already described above)? In a separate web page & docset? For the console help functions, will they be in a separate "category" (we currently have hs, spoon, and lua as the top level categories, and I'm pretty sure with this last rewrite I made it so it that adding new top-level categories is pretty simple, but I'd have to review hs.doc to be 100% on what needs to be done).
• Hmmm... I had a third one in my head, but it must have fallen out of my ear hole... if I recall it, I'll add it later.

@cmsj, I think a certain amount of cross-module mixing is inevitable (after all the point of helpers is to wrap common reusable code and some of the modules are pretty specific in what they're able to do by themselves), but I do think that in most cases there will be one that stands out as the "primary" with the others providing a supporting role. This also poses the question, does a helper have to be specific to a module? Could it provide its own "category" of functionality?

This could quickly turn into a long dive down a deep rabbit hole... I think we need to clarify what we consider a module, a helper, and a Spoon. If I have an idea for adding some functionality, how do I decide where it belongs? I know that a certain amount of "we'll know it when we see it" is going to happen, but clarifying, even if broadly and generically (even if it's something as vague as "if it requires it's own compiled code, it can't be a Spoon"), what each should contain and what they shouldn't, it will help.

I need to clarify my own thoughts on this a little before putting down what I kind of think the boundaries for each might be, but feel free to add your own thoughts (or tell me why I'm thinking in the wrong direction) if you wish.

---

tl;dr -- this is what I was writing when I decided to just add the last two paragraphs above... it may shed some light on my thinking or it may just confuse you... read it only if you want to.

The kind of helpers I had in mind with my original musings kind of fall into two categories... and maybe we should clarify whether or not both categories fit the consensus.

1. Useful functions that encapsulate some logic which lends itself to easy reuse in slightly different contexts, but doesn't necessarily provide something "new" itself... If it didn't exist as a module already, I'd say hs.fnutils would be a good candidate for this type. As are the functions in CommandPost's cp.axutils (especially the ones that can determine which child elements are in the same layout row/column as the one you pass to it).
2. Targeted interfaces that don't require a lot their own unique infrastructure but that don't necessarily provide something a user can just load and run, like the Spoons do... these might almost justify being modules themselves, but don't require much, if any, code that doesn't already exist in one or more modules... as specific examples of what i'm thinking about here (and at some point I'd like to add them both to Hammerspoon, but they need a lot of cleanup and documenting first):
   1. I have a pure Hammerspoon Lua (meaning it does use some of our modules, but doesn't require new compiled code) "module" that I wrote to interface with the Philips Hue bridge for home lighting control. It does nothing but attach to the bridge (or prompt you to press the button so it can) and then provide HTTP Get/Put wrappers around JSON code to query and control things. By itself, it's just an interface... you'd need more code to create hotkeys for specific lights, any kind of visualizer, etc. and those definitely sound like Spoon ideas... but you'd need this to exist first.
   2. likewise, I have a Roku control "Spoon" that provides similar for Roku devices on the network... and a separate RokuRemote spoon that actually displays a remote on your screen with hs.canvas and allows you to press buttons and control your Roku... that latter belongs in a spoon, but the former... doesn't "do" anything by itself and I've never been happy with it as a Spoon, but it also is completely written in HSLua so I'm not 100% convinced it should be a module either.

As I'm writing this, it occurs to me that with category 2, I'm really trying to ask what do we want to consider a module? When Spoons were first introduced, it had been suggested (I don't recall by who or in what specific context or conversation) that if it can be written entirely in HSLua, it should probably be a spoon rather than a module. I'm not entirely sure I agree with that, but I also think we've got a number of "modules" that are such similar variants on a theme (and in the case of most of the music app ones, that even use the same core with a slightly different top-level "wrapper") that in hindsight I'm not convinced they should be "modules" at all but rather something in between. But is that in-between what we're talking about when we speak of helpers here or is it yet something else?

[randomeizer]

The vast majority of our modules are pure Lua, and import multiple hs (and cp) modules.

The line can be fuzzy. For me, a plugin/Spoon is typically a thing that a user might install for a specific purpose. It would usually have some user-facing functionality in and of itself.

That said, we don’t follow that rule strictly for our own plugins - there are several ‘manager’ plugins that are dependencies of multiple other plugins, along with some things like ‘menu’ categories, etc. These could (should?) be modules by my earlier definition.

Helpful? Probably not. :P

[latenitefilms]

FWIW - I like @cmsj's idea of just keeping things simple, and adding a hs.$MODULE.helper module for each extension. This would mean you can use the existing documentation infrastructure, and the existing scripts to bundle the new modules as part of future Hammerspoon releases.

You could also add hs.helper for more generic helpers that span across multiple extensions.

I can imagine there's lots of things in cp.tools that could be applied to various extension helpers.

As @asmagill notes, I also think there's probably stuff in cp.ui.axutils that could be added to hs.axuielement.helper.

To me, "helpers" are more handy Lua tools that allow you to develop faster, whereas "extensions" are the lower-level stuff (i.e. Objective-C code that interacts with macOS Frameworks) and "spoons" are essentially plugins that users can install to add new features to Hammerspoon with minimal coding required.