Fake/virtual input

[tinne26]

Hey, it doesn't seem like there's any feature for faking or reproducing input programmatically. I know this was just released a few days ago, but it's looking so good that I had to ask! Are there any plans or desire for this?

If you consider this outside the scope of the project, please feel free to close the issue.

[quasilyte]

Hi!

This could be useful for sure. I wonder what kind of the API would be better.

Right now we assume that *input.Handler can only check the input state: whether some action was activated or not and get some events info (if any). There could be multiple actors in the system that use the same input handler to check for the inputs (UI elements, scene objects, etc).

It can be frustrating to see the ordering issue is one of the handler users emits some input event, but not everyone can see it (only the ones that are processed after the event has been triggered). Perhaps this can be solved by making the emitted action only be visible during the next frame (duplicated action triggers will be ignored).

What other strategies there could be? Do you have a usage example for the context?

[tinne26]

While a new VirtualInput as an InputDeviceKind could be considered, the most basic approach would be probably something like System.QueueVirtualAction(Action) or something along those lines (e.g.: QueuePress / QueueUnpress). Would probably need some additional EventInfo or others if touch and other more complex events are required, but yeah, having it become visible only after System.Update seems fair. For testing that works, and for input playback, you can also pass it during Game.Update before you update the input system.

Some more thought and details are needed, but I generally think that queueing the virtual input and have it become visible only after updating is reasonable, as we have control over when we call Update() on the input system.

[tinne26]

I have other priorities at the moment but I'm still thinking about this. Some minor notes in the meantime:

• We definitely need "virtual actions/input" to be able to simulate something like touch and other more complex input events. Any kind of input event should be emulable: if a game works only with touch and needs to store input for replay, this is definitely necessary. Just "virtual key presses" won't really be enough in the long run.
• There's another important case that I kinda forgot about: input coming from remote connections. Examples are multiplayer games and games that build their own input mechanisms (e.g.: using smartphones as controllers in a LAN game). This means that we should avoid creating an API that sounds like it's made only for testing and input replay, and we should probably support an unlimited number of "virtual input sources".

[quasilyte]

Good points. :+1:

[quasilyte]

https://godotengine.org/qa/5896/simulate-mouse-and-keyboard-events-by-code Just for an example of how it's done in Godot.

[quasilyte]

I provided an experimental API for this. I think it should be good enough for the most use cases.

I'm not sure it addresses the "virtual action" part though. It just allows a Key (which could be a touch/click/keypress/etc) event to be simulated. As long as you have some way to associate them, it should be OK.

It should be possible to simulate the action event too. There is a question though: if there is a list of keys that could activate that action, which key should we choose?

[tinne26]

Looking good! I agree that being able to simulate actions directly would be relevant to avoid depending on specific keymaps for testing and replays; it seems cleaner. With regards to your question, if actions can be simulated, wouldn't that be the cleanest solution and remove the problem? Otherwise, I think use-cases where the specific choice can be relevant are fairly limited... maybe if multiple actions can be triggered by the same key in the same context (I don't know if there's any check for that, but I can't imagine why anyone would do that intentionally) or macros and key combinations exist, in which case unintentional collisions could happen, I don't know if that's a future concern. There's KeyWithModifier too, but I'm not sure if that can actually cause problematic collisions under the current implementation. I think I'd only be using an actual key instead of an action if I'm mimicking something the user did, in which case I'd use the exact same keys used.

[quasilyte]

I agree that for the testing purposes the action activation is cleaner than the exact key method. It looks like it shouldn't be a problem to add an action-based event emulation. I'll try doing that tomorrow. :)

[quasilyte]

I think I'm pretty happy with the final result.

Now it's possible to trigger the action directly, even if no keys are bound to it.

Right now the events will be visible only to the handler with the same player ID. This is identical to how gamepad-like events work (they're device ID bound, which is identical to the player ID). It's possible to add a broadcast version of this later, if there is a need for that.

Another consideration is that the keymaps can be configured by a player, so it's not always convenient to check whether some key is assigned to an action or not. So having an action-based API here is nice.

[tinne26]

Looks good to me too. All the main use-cases discussed seem to be properly covered now, so I guess we are good to go!