RFE: create a shared key macro capability

[i30817]

Description

If the creation and saving of macros per game (it really only makes sense per game so you can save automatically there) was possible in retroarch, a bit like zsnes, some games would much easier to do cool things on.

Expected behavior

A menu like zsnes had where you could record a macro. If i recall correctly, zsnes presented a menu with all possible keys and allowed you to click them into a sequence, or click on a 'wait' marker which would get presented on a label.

While this is easy to use, in retroarch (which often does not support mouses in some configurations and where long labels are not very usable) i propose using key capture + a optional post-capture edit step instead, which also has the advantage of being able to incorporate analog movements optionally. This has the disadvantage of being harder to use (a skill step first) and maybe wasting some time if you can't get it right the first time or edit the macro to minimize the time it wastes.

1. On the input menu you should be able to assign 'real keys' to macros, which do nothing until connected to a existing macro on the new macro submenu. These keys would be 'special' and if you keep pressing them they do nothing until the frame after the macro 'ends'. Similarly, if another game key is pressed, the current macros are interrupted. In short, there can be 1 or more macro running but if a normal non macro key is pressed, they all get interrupted.
2. On the macro submenu you need to be able to create new macros and be able to connect them or not, to a existing macro key. Here you can allow two macros to fire at once from a single key.
3. On the macro submenu, present a image of the controller/retropad with the buttons shown, these would show, with a macro selected but not on edit mode, a loop the macro on the controller image, so the user can see it work. Slowed down ofc.
4. If you're creating a macro wait for user input, capture all of it and then subdivide into into a 'time macro unit' (it's easier to build it all at once and then modify it). Keeping 'X' button pressed for more than 1 second : exit macro capture You probably want to not record empty 'time units' after the start and before the end.
5. for modification, select the macro, and then press x to edit and get presented with a list of 'macro time units' where if you select them you can edit the 'unit'. You can show the toggled keys on the controller image and the time unit number (for example 1 / 20, first of 20 macro time units): Keeping 'X' button pressed for more than 1 second : exit macro editing Keeping X_Axis button pressed for more than 1 second : move to next or previous unit Keeping circle button pressed for more than 1 second : delete unit. Tap keys just toggles the button to the macro.
6. Present the information about the key sequences chosen to control the macro on the help label (since it's very different from the usual input mapping 'single key fire and forget'). I feel that the 'hold longer' to control is intuitive and the exit behavior being similar will remove most user error, but if you want you can show as text 'Press X 1 second to exit' when input capture is open. Blind people trying to play games... would read the menu descriptions, though they wouldn't be able to get the visual feedback without extra TTS interface effort that i don't know if retroarch supports.
7. You need to be able to delete macros on the macro submenu. 'clearing' should remove the macro key assignment (essentially the same as disabling it) so it can't be the same operation.
8. You need to be able to name macros for ease of recognition.

Some keys are probably illegal to press at the same time (like same axis arrows for example) but those are details.

You probably would be able to reuse the 'key sending' capability of multiplayer for injecting a macro on the controller.

I don't know if this is the best design, and it's probably a lot of work, but it seems a cool idea, so here it is for posterity or glory.

Alternative: no gui macro creation

Just create a protocol to write macros manual and a function paste them from the clipboard into retroarch (could be done manually by editing config files like cheats too, for platforms without a clipboard). This is basically the simplest implementation that would still be effective with sufficient 'popular support' imo. Leave it to the hobbyists to learn to write the macro and share it widely and easily, instead of trying to make a GUI for all users to write them themselves. Making analog gestures work with this is likely to be complicated (probably as a set of splines and a start and end number for the units they apply) and grow the string a lot (unless the splines are represented as formulas but that's probably too complicated in many ways).. Probably the least buggy implementation and quite a good idea if you want to scale up later because a good foundation here would be good for a scale up.

Unsolved Questions

should be 'creating a new time unit' actually translate to a single frame?

I'm not sure. Sure most games will react well and blazingly fast to that, but some will either detect that as a impossible input (a certain scene in metal gear solid comes to mind) or require more of the same input for significant amount of frames, which could get annoying to edit. Maybe it could be a set time value like 'at least 0.1 seconds'?

sharing macros

It could be interesting to make macros have a string representation that can be copy pasted easily for sharing in the internet or IM. You'd need a button to 'copy macro into the clipboard' and another to 'paste macro in the clipboard'. Later, extending libretro-database with macros for individual games could be explored if the number of player made macros grows a lot (as i expect it will if a copy paste feature is made available). This would be especially interesting for wii games that are 99% playable except some complex analog gestures if a analog control recording option is created.

Multiplayer cheating and achievements

I don't think you should do anything except disable achievements or show to others in the lobby that they're using macros. To cheat that they'd have to recompile retroarch, which isn't hard, but Joy2Key macros aren't hard either and also undetectable.

Analog Controls

Analog controls are ... difficult. Gestures, which are the typical analog control problem don't map very well to the button interface, because analogical movement and digital button presses operate on a entirely different scale. This might be compatible if a analog movement operates like a spline subdivision over a interval of 'macro time units' and the splines are actually running per frame instead of losing most of their data. In the following ideas too, 'creating' or 'deleting' a time unit when editing a macro may cause problems, and i'm not sure if they'd be significant.

Alternative 1: gesture library

Oriented for the most common types of analog controllers, the sticks and the most common of gestures (circular motions) and show those as 'preset' analog macros.

Namely: full circle starting at XYZ extreme axis clockwise (4) full circle starting at XYZ extreme axis counterclockwise (4) half circle starting at XYZ extreme axis clockwise (4) half circle starting at XYZ extreme axis counterclockwise (4)

These presets would naturally be hidden on a the retroarch equivalent of a sub menu on the macro submenu. In there you'd be able to assign some of these to 'keys' that you'd only be able to use on input capture. If these 'keys' are held, they don't toggle the controller key in the macro (if they correspond to a controller key, they could be keyboard keys without a controller mapping).

Then in 'create a macro' and you'd tap whichever 'digital' buttons you need, but when you need the 'analog' stick movement keep pressed a special key for 'N seconds'. Unlike the exit key, this key would be a marker of 'now i'm executing the movement for as long as i keep this pressed'. You could repeat this or press more 'analog keys' at the same time.

In the final macro, the speed of movement of these sequences would be adjusted so it lasts exactly 'N seconds' from start to finish.

This doesn't really scale (dual shocks would multiply this already long list by at least two, assigning keys would be annoying etc), and only takes care of the circular motions case. However, it takes no skill or the device to create.

Alternative 2: complete gesture capture

In this alternative the dropdown box doesn't exist and the button really does capture all input (could be multiple analog axis at the same time, wiimote motion, normal digital button etc) and it would exit the same way.

This would be flexible and work for complex gestures but it would be (much) harder to have the skill to create one - and impossible if you don't have the device, like a wiimote. The later is a 'problem' because it's people without the devices that most need the feature. Without a 'very easy' way to share game macros this is a paper tiger feature.

Proposed GUI

Notice that these are already a great many extra features that require editing macro properties, edit macro time units, naming, copy into clipboard, copy from clipboard, delete macro and, if you want to implement the gesture library alternative a large list of analog stick gestures that can be bound to multiple keys.

This is my proposed structure.

macro submenu: image of the controller with looping Unit keys pressed shown if a entry on the last list is selected create macro button, captures keys like proposed paste new macro from clipboard button open assign analog gesture submenu button (if the gesture library is created) __list of created named macros (with default names if not edited yet)

the list of macros could be assigned to a 'macro key' with left-right arrows, like you can for dropdown boxes equivalent in retroarch, or cleared with the clear key to disable them.

open assign analog gesture submenu button would open __list of all gesture presets with a key assignment or none (can be reset to none) that only works on the create macro button if held to record the number of seconds and thus the speed of a 'gesture' preset. This can be a usability problem if people are trying to create macros on controllers and need change keys assignment when creating multiple macros because they want both the analog movement and the controller key.

For each macro in the last list, selecting it would show: rename macro label copy macro into clipboard button delete macro button list of the macro captured 'time units'

selecting unit from last list of the macro captured 'time units' would change controller image to show the keys at that time unit (analog stick position if any should be shown, not sure if wiimote gestures can be shown meaningfully) On select, open a version of the key capture that doesn't advance time and you have the extra 'held button' functions in 4. above. I think it's better to just prevent analog controller movements from being changed except on the main capture widget, because it allows you to not care about things like deadzones or 'snap to center' controls screwing up your edition of the time unit.

The proposed GUI for the main alternative without macro creation is basically the same except without all the options that are no longer needed (button capture, list of time units, copy macro to clipboard, assign gesture submenu).

Actual behavior

Nothing, feature is not available, though it probably could exist on all cores, in emulators that never had the capability.

Steps to reproduce the bug

Bisect Results

Version/Commit

Environment information

[i30817]

If some one read this already i edited the macro capture section because i had a better idea than key combinations to control it (keeping buttons pressed to control, quick tap to select).

edit2: added the idea of 3d spline capture for more complex controller gestures. Not sure if it would be made to compose with the per 'frame/unit of time' that is the main idea of the RFE.

edit3: clarified a bit and removed some bad ideas from the analog controls case.

edit4: removed the 'sharing macros on the multiplayer rooms' musing. That would happen anyway probably but much more important is to be able to copy past the macro into and from the clipboard so communication can happen and people can just plop down macros on forums.

edit5: a better idea for 'macro creation' where you first create the macro completely by inputting it, it gets divided into 'macro time units' by retroarch and then you can edit those units. This is better because splicing analog movement would be very tedious otherwise and can work for both the preset library idea and the complete capture idea.

edit6: clarified some more, removed the 'create a new time unit' on the macro edit idea. It's easier to just delete the macro completely then try again (especially with analog gestures into the mix).

edit7: removed a idiot notion again on the analog capture idea.

edit8: add a explicit gui structure proposal.

edit9: add a alternative where just a 'protocol/language' for creating macros is created and a paste function.

[i30817]

Ok, i got what's probably a better, less ambitious idea. I wrote it up there in 'Alternative: no gui macro creation', but if it's tl;dr, basically instead of trying to make users able to record and edit their own macros, create a protocol describing a macro (and splines operating on macro intervals for the analog movements with velocity derived from delta time of the interval and spline distance) and a paste from clipboard button.

That's it, leave the nerds to create the complicated spline wii-mote movement macros, or reasonably diligent people that can chain together a list for the normal macros and leave the lusers to get them from copy pasting from the internet and make sure you got your de/serialization and implementation of 3d splines and runtime injection of keypresses right.

This would be a good first iteration because getting this right could be seen as a bottom-up step for the more complicated input capture the rest of the post attempts to describe... and it certainly would be created faster than a complicated GUI...

Much less likely to segfault retroarch wouldn't you agree?

[andres-asm]

The input remapper would certainly allow this to happen.

But it is far from trivial to implement. Combos may be achievable within a reasonable timeframe (sans GUI) but this is in a whole different level of complexity.

[i30817]

I agree, but i'm rooting for the limited version i put in my last post. You don't really need to make a GUI for the functionality to be useful, just for opening the door to 'anyone' creating one without a text editor or serious patience (in the case of analog complex movement).

If you create a parser that takes something like: "Ryu hadoken" [down,right-down,right,x]

and creates a macro entry, that would be a very good first step. If you enhance that with the equivalent analog:

"Ryu hadoken" [{ thumbstick down-right diameter spline, 0.2seconds}, x]

or even

"Ryu Shoryuken" [{ thumbstick right, 0.1seconds }, {thumbstick diagonal left-right, 0.13seconds}, {thumbstick right, 0.1seconds}, x]

I'd be ecstatic because it would be a large step for more complex curves for wiimotes (that would need rotation, much more subdivided curve segments, multiple axis of movement at the same time (for dualshock or 3d) etc - basically i'm trash at maths so don't expect this particular protocol idea to be good).

Even with just a parser and config file settings the above would be nice, but if you implement this far, having a button to try to parse the first string in the clipboard is not a step too far for usability and i feel would make this very popular on pc.