DirectInput support

[EmanueleCiriachi]

Greetings,

I am an Elite Dangerous player and Flight Instructor in my squadron; I used tools in the past to teach other players how to fly without Flight Assist, in particular GamepadViewer.

Nowadays however, I switched to mouse, keyboard and rudder pedals, and I struggle to find a solution to display all these inputs on screen.

Conveniently, I am also a .Net full stack developer.

My question is, how hard do you think it would be to customize NohBoard to achieve the following:

1) Customise the keys displayed on the overlay; 2) Add support for DirectInput peripherals.

Kind regards, Emanuele Ciriachi

[ThoNohT]

I think this is related to #22?

I have no experience with DirectInput or XInput. I think from the tracking and displaying point the folowing things are needed:

1. There may need to be a whole new type of key-codes. Currently we have mouse codes, which I have as an enum, and keyboard key-codes which are just ints, 0-255 are the regular key-codes, and then I juse 1024 and up for some custom keys like enter, and the lock-key states. These could be DirectInput keycodes, and I guess they are also numerical? But of course, they aren't all binary. So aside from the KeyboardState and MouseState class, a DirectInputState class can be added that provides the information about the states of the keys there.

2. There needs to be a new set of elements. Currently they are split up over Keyboard Keys, Mouse Keys and the Mouse Speed element (which justt has its own tracking code, but is still in MouseState). Then there would be a DirectInput key/button, a toggle? And analog inputs that have a a value of 0-255 for example? They need their own rendering and customization code.

---

Then there's the point of hooking and detecting the input. I have no experience with it. But I guess that information is available somewhere. I guess it will be a matter of selecting the device to hook, and also plug in message loop?

I currently don't have the time to work on this. But if you want to pick it up, I am always willing to review the code and help where needed.

[EmanueleCiriachi]

Hey Eric,

I'm actually working on it on these days. I have a simple prototype that polls the specified DirectInput device every x ms (I could not find another way of detecting inputs) and returns the data either in an immediate or buffered fashion.

I was precisely looking at the StateBase class and how to create a JoystickState or DirectInputState.

I'll keep you up to date with my progresses.

[EmanueleCiriachi]

Ok, I spent almost the whole afternoon / night coding, and have come with a sketch of an implementation.

My current problem is that, after defining a DirectInputElementDefinition and a DirectInputButtonDefinition, these are not parsed.

Is there any way to troubleshoot FileHelper.Deserialize<KeyboardDefinition>()? As in, something that tells me why my JSON is not parsed rather than just omitting it.

[ThoNohT]

I'm not sure of any way to really debug this JSON serializer. However. Is this the code you are working on: https://github.com/EmanueleCiriachi/NohBoardBranch/commit/0b351c14b530d4ba720e9eebc6646b6acaf3a0f0 ?

In that case, I think you're missing two things that prevent parsing:

1. You need DataContract attributes on the classes you want to parse, like this: [DataContract(Name = "KeyboardKey", Namespace = "")]

2. You need to let ElementDefinition know it has more subtypes, like this: [KnownType(typeof(MouseSpeedIndicatorDefinition))]

Then finally, on all properties that must be serialized, you add a DataMember attribute.

[EmanueleCiriachi]

Thanks a lot, I was about to have another shot at it.

I'm not much of a Forms person myself - I'm more web-oriented, but I really like the way you structured your code, clean and professional.

Finally, I could not find a way to poll the joysticks via events; allegedly a hook exists, but I could only find a C++ version of it: https://docs.microsoft.com/en-us/previous-versions/windows/desktop/ee417927%28v%3dvs.85%29

[ThoNohT]

That's what you can use p/invoke for: https://docs.microsoft.com/en-us/dotnet/standard/native-interop/pinvoke

It is also how I create the low level keyboard and mouse hooks.

[EmanueleCiriachi]

I'm not familiar with pinvoke; after everything else is sorted I will try my hand to it; in the meanwhile I will use the timer-polling as a temporary solution.

[EmanueleCiriachi]

Ok, parsing works now. My new elements however, are not being rendered even if I'm using the standard style definitions for KeyboardKeyElement.

Where exactly in the code is an Element associated to a rendering style?

EDIT: nvm I found it.

[EmanueleCiriachi]

Ok, I posted in my branch a first "working" version.

Double air quotes because it's very unresponsive. Is this a consequence of my timer-based polling strategy? And please disregard the "EmaKeyboardKeyDefinition", that was a test I made.

[EmanueleCiriachi]

The annoying thing that now must be done manually is specifying the GUID of the DirectInput device you are using in the DirectInputButton configuration. Also, no axis support yet - that will come later, after I have a better foundation.

[EmanueleCiriachi]

Never mind I fumbled - with a polling interval of 40ms it's relatively efficient! I'll start cleaning it up a bit.

[EmanueleCiriachi]

I have a basic implementation for axis! Next is the d-pad, then it's cleanup time and I will study how to create custom styles.

[EmanueleCiriachi]

DPad is now done - off to understand how styling work now.

Side-question: can this work as an overlay?

[ThoNohT]

I don't understand the question I'm afraid.

[EmanueleCiriachi]

I mean, can I have the graphics that are generated by NohBoard appear as an overlay over my game, and be part of the desktop recording without needing to use OBS?

[ThoNohT]

Not out of the box. But I guess it can be programmed. But that's not a feature I intend to have in NohBoard, because the implementation will vary wildly on what game is running, which engine it uses, etc.

And if keyboard hooking is not enough, that's another reason for NohBoard to be considered a cheat, or a virus by games and antivirus software.

[EmanueleCiriachi]

I see. Would you be ok with me making a fork of NohBoard, which includes my DirectInput support, that acts as an overlay? I will make sure that it works for Elite Dangerous as this is my end goal.

[ThoNohT]

Sure, but I do hope you plan to create a pull request for the DirectInput stuff, as that I won't mind merging in. I will want to do some reviews to make sure the code is up to the standards I hold for NohBoard, and the user interface is easy to use, etc.

Other than that, it's open source, so everyone is free to fork and do their thing with it, as long as they abide by the license.

[EmanueleCiriachi]

Of course I will create a pull request. I just want to make sure that the style is working correctly first.

There are two issues that I am unable to deal with in the short term:

1) I could not configure p/invoke to react to Events raised by DirectInput; my implementation polls the status of the device once every 40ms.

2) The configuration file relies on the user knowing the GUID of the Direct Input device, which is not the most intuitive way to go about this. I'm not sure how to address this - perhaps I should provide either the GUID, or an ordinal number that will pick up the n-th Direct Input device it detects.

Beyond this, it works fine.

[ThoNohT]

Can you ask for a list of DirectInput devices, and put them in a dropdown somewhere?

[EmanueleCiriachi]

Yes, but a mapping can involve more than one DirectInput device, and this mapping is implicit in the config file. I could count how many different devices are mapped in the config, and then have a dropdown asking you to map each and every device specified in the config. But where should this dropdown appear?

[ThoNohT]

I don't know those answers either right now.

They could also be in the element definition properties form, I guess. But then, it is still not easy to have an easy way for everyone to always use this, since they have different device GUIDs than the one who created the keyboard. But if this is going to be available to everyone, then everyone must also be able to load a keyboard and map the elements to his device.

Perhaps some way to do is, is to ask a user to press the button they want to associate with each element. This allows the user to map every button to a random key on a random device. There will just need to be checks that the different types of elements match up.

[EmanueleCiriachi]

I created three new Elements: one for a DI button, one for a DI axis, and one for a DI directional keypad.

Do I have to create Forms to edit their properties? If I don't, will this break the application?

[EmanueleCiriachi]

Custom style and custom icons - this is more or less how I intend to use it:

https://i.imgur.com/hVoog6E.png

[EmanueleCiriachi]

Bear in mind that axis can also be bi-dimensional.

[EmanueleCiriachi]

Hi, I'm still alive - sorry for the prolonged silence.

I'm working on making my new controls editable, but I have a problem; in the btnDetectButton_Click event of DirectInputButtonPropertiesForm, I cannot access the "SelectedValue" field of the ComboBox where I have all the available Buttons. That property is available during/after intialization, but during the

public static Func<int, bool> DirectInputButtonInsert

function called in HookManager.Public.cs that field returns an error. Why does this happen?

[EmanueleCiriachi]

I updated my code into https://github.com/EmanueleCiriachi/NohBoardBranch/commit/0b351c14b530d4ba720e9eebc6646b6acaf3a0f0

[ThoNohT]

I checked out your code. But I can't see the DirectInputButtonPropertiesForm. Are you sure it is committed?

[EmanueleCiriachi]

It wasn't - it is now, sorry.

[ThoNohT]

Oh I see it now. It is one folder up too far though. This probably happened when I migrated from .Net Framework to .Net Core, I moved the entire solution folder down one level.

I'll move it to the proper place manually to do the testing for now. But just so you know

[ThoNohT]

I'm not managing to trigger the detection. I'm not sure how to assign it a controller at this point, I could try with my Xbox One controller.

But, judging from the code, I think the following thing is going on: In this code:

           Hooking.Interop.HookManager.DirectInputButtonInsert = (code) => {
                    this.cmbButtonNumber.Text = code.ToString();
                    return true;
                };

You are doing the detection on another thread, namely that of DirectInputTimerProc. This can be resolved by explicitly moving all UI interaction back to the UI thread. Some searching shows me that for Windows Forms, you can use this.BeginInvoke for this: https://docs.microsoft.com/en-us/dotnet/api/system.windows.forms.control.begininvoke?view=netcore-3.1

So, the code would become something like this:

            Hooking.Interop.HookManager.DirectInputButtonInsert = (code) => {
                this.BeginInvoke(new MethodInvoker(() => {
                   this.cmbButtonNumber.Text = code.ToString();
                }));
                return true;
            };

At least this compiles for me 😃 I can't test if it works.

[EmanueleCiriachi]

Sorry, back in the saddle and I'm already having problems even going back to where I was (I moved my machine in the meanwhile).

How can I properly build the application? The bin\Debug\netcoreapp3.1\win-x64 folder contains 298 files, including a lot of DLLs; the .exe of the application alone is only 171Kb against the 825Kb of the release - this was not happening in my previous setup. If I run the .exe there it works, otherwise it complains about missing .Net Core.

[EmanueleCiriachi]

As for your question, the reason it doesn't find your device is that its ID must match the Devide ID in keyboard,json.

Currently, there is no user friendly way to find it - just add a breakpoint in the AcquireJoysticks() and retrieve it from device.ProductGuid. Eventually I will a dropdown of sort.

[EmanueleCiriachi]

I'm almost done with the Forms portion! I just need to test the DPad component, but otherwise this time I'm really close.

[EmanueleCiriachi]

It is done! @ThoNohT , I have the first version ready - complete with style and editing support for the new functionalities.

Give it a whirl, then tell me if I can try merging it in a branch on your repo.

[EmanueleCiriachi]

https://github.com/EmanueleCiriachi/NohBoardBranch/releases/tag/0.8

[ThoNohT]

It crashes right as I start it. But I don't have any DirectInput devices connected right now.

[2021-05-24 08:44:55] System.NullReferenceException
[2021-05-24 08:44:55] Message: 'Object reference not set to an instance of an object.'.
[2021-05-24 08:44:55] Stack trace:
[2021-05-24 08:44:55]    at ThoNohT.NohBoard.Keyboard.ElementDefinitions.DirectInputAxisDefinition.Render(Graphics g, Dictionary`2 directInputAxis, Boolean shift, Boolean capsLock)
   at ThoNohT.NohBoard.Forms.MainForm.Render(Graphics g, ElementDefinition def, List`1 allDefs, IReadOnlyList`1 kbKeys, List`1 mouseKeys, Dictionary`2 directInputKeys, Dictionary`2 directInputDpad, Dictionary`2 directInputAxis, IReadOnlyList`1 scrollCounts, Boolean alwaysRender)
   at ThoNohT.NohBoard.Forms.MainForm.OnPaint(PaintEventArgs e)
   at System.Windows.Forms.Control.PaintWithErrorHandling(PaintEventArgs e, Int16 layer)
   at System.Windows.Forms.Control.WmPaint(Message& m)
   at System.Windows.Forms.Control.WndProc(Message& m)
   at System.Windows.Forms.ScrollableControl.WndProc(Message& m)
   at System.Windows.Forms.ContainerControl.WndProc(Message& m)
   at System.Windows.Forms.Form.WndProc(Message& m)
   at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
   at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

[EmanueleCiriachi]

Hi, sorry for replying this late, I had missed the notification.

I updated the release, the crash was caused by failing to update the baseline configuration. Would you mind to try again?

[Thedoczek]

Hi, sorry for replying this late, I had missed the notification.

I updated the release, the crash was caused by failing to update the baseline configuration. Would you mind to try again?

I'll try with XBox pad

[EmanueleCiriachi]

Hi @Thedoczek , let me know if you need any help.

I even made a video that shows how to configure it: https://www.youtube.com/watch?v=kkXQnSVomUA

[Thedoczek]

Hi @Thedoczek , let me know if you need any help.

I even made a video that shows how to configure it: https://www.youtube.com/watch?v=kkXQnSVomUA

Nice Imo each of DirInput option you presented should have ability to select square or circle background

[EmanueleCiriachi]

I see - I don't think I will pick it up again to add purely cosmetic changes though.

[Thedoczek]

@EmanueleCiriachi There's one critical issue for me: For Xbox One controller I can assign left joystick, but I can't assign right axis.

[EmanueleCiriachi]

The Axis doesn't appear in the list of Axis? I tested it with a PS3 controlled that emulated an XBox controller and it worked fine. Could you post some screenshots?

[XavierShrier]

@EmanueleCiriachi I know alot of time has elapsed but do you ever plan to merge your fork? It'd be really really helpful for me and would expand the program alot. I don't think main has had major changes since you worked on your fork either, so it might not even be too painful if you wanted to try 🤞.

On another, related note, I tried installing your fork and it spits out a prompt to install .NET Core. From my limited understanding of things the naming system for .NET/.NET Core/whatever is a mess, but I do think I have the latest version of .NET installed, per Microsofts download page. Is there something I should do to fix this, or it it just the program being out of date or some such. If it's the latter, would you be willing to fix it (even if you don't have the time to commit to a full merge)?

Happy to provide more info and test stuff, I'll keep an eye on this thread for a while.

[EmanueleCiriachi]

Hey! It's been a while indeed.

Guess what? Today is bank holiday here in Britain and I'm not terribly busy for a change, so I might try to do just that.

[XavierShrier]

Update: Managed to get it to launch after looking over the files to find the one that defines the version it wants and downloading exactly https://dotnet.microsoft.com/en-us/download/dotnet/3.1. I don't know how much me also not installing the x64 versions of other .NET stuff matters but up until I saw the config file I had been using the x86 ones instead, and the button to download it didn't work for me so /shrug, going to call it user-fixing instead of user-error.

[XavierShrier]

Issues I've found (in rough importance):

Important:

• .NET either needs to be bumped alot or have better pointers to where to download it, since it failed to load a page for me.
• The application will crash if (I think) using an old style when viewing/adding/whatever a Axis or DPad input. A warning/prevention/whatever as a fix is probably fine.

Less important but probably easy and would be nice:

• Buttons need a toggle for the always-a-circle behavior; the current way is cool but I would prefer to be able to make them look like normal keys/have custom geometry.
• The Axis and DPad elements should have transparency by default instead of a black background.
• The Axis element should have a default option to make it a circle, not square, and to remove the "glow" effect. (This sucks for chroma keying).
• DPad directions should have an option to represent as buttons individually (ie having an element that activates when I press right on the dpad, instead of the 3x3 square).

Minor naming shit:

• The Change Text on Caps + Shift Text options should be removed.
• The window itself needs to be renamed from Keyboard Key Properties to Direct Input <Type> Properties.

I'm aussie so I wanted to post this then sleep. Enjoy whatever you decide to do!

[EmanueleCiriachi]

Thanks, I'll keep those in mind.