Keybinds for Neverwinter

[Aosys]

Hi there, I'm here with something of an unusual request. Neverwinter is just getting the bard class (it's not even live yet, currently still on the preview server), and I'm interested in having software than can convert midi files to the right keypresses in-game to play arranged music. After some light testing I found that TBbard does indeed register keypresses with NW, but NW's note configuration is different from FFXIV's, and I'm having trouble figuring out to adapt it (my Java is VERY rusty).

We get 3 octaves, using left and right arrow keys to control them, and sharps/flats are controlled by up/down arrows. We also don't get the full keyboard layout like FFXIV sadly, so we're stuck with only numbers 1-8 for notes, no letter keys. Therefore, high e# ends up being a combination key press of 4, up, and right for example. I noticed there was a to-do note in the Notes file about making keybinds configurable; was this ever implemented?

[isalin]

This was never implemented, unfortunately.

I might be able to look into it though!

Is the bard perform system available from the start in neverwinter?

[Aosys]

Yes and no. Bard isn't live on the main servers yet, and I'm not sure precisely when they intend to release that update, so for now it's only on the preview servers for us to play around with. There are extra steps to get onto preview, but it looks like it's possible to get on with a brand new account (I just tried it). The steps are:

1. Download Neverwinter and create an account.
2. Log into the game on live and create any new character. None of the stats matter, this is a throwaway. Finish character creation and get into the tutorial. You shouldn't have to go further than this.
3. Exit the game, then log into this link using your Neverwinter account: https://www.arcgames.com/en/sign/in?src=http%3A%2F%2Fwww.arcgames.com%2Fen%2Fmy%2Fcharacter%2Fcopy%2Fnw
4. Accept to gain access to the preview server, then copy your newly-made character over (I chose Owlbear for my server, not sure if that makes a huge difference)
5. Once you've copied your character, open the launcher again and log in. At the bottom, to the left of "Play", you can now select "NeverwinterPreview" as the server you want to access (it's a big download, be warned)
6. Make a new character on preview, making sure to select Bard as your class
7. Actually complete the tutorial with this character, then go into the Adventurer's Guild and talk to Sergeant Knox. You don't unlock the music function until after you do this.
8. You use tab to get into play mode when you do unlock it

These instructions are somewhat involved, so I don't blame you if you don't want to go through them...

[isalin]

Wow, thanks for the instructions!

I'll check it out after work.

[isalin]

I've attempted to make a version for Neverwinter (using the default keybindings).

Would you mind helping out with the testing? Neverwinter.zip

The UI is still using the old FFXIV one. So make sure to disable "Use full keyboard layout".

Personally, I felt like most songs sounded a bit weird when you got to the upper and lower ends of the octaves. Some songs sounded a lot better when you adjusted the octave target, and som ejust never really sounded good. But I think that might just be the way the notes sound in Neverwinter... unless I did something wrong :laughing:

[Aosys]

First off, I can't thank you enough for this, myself and my fellow would-be Neverwinter bards really appreciate it!

I just ran some songs through and it seems to work great! I'll have to play around with creating some custom midi songs to see how to work around Neverwinter's limitations (e.g. you can't play chords with both naturals and accidentals unless you arppegiate them), but so far so good!

And yeah, some of the NW notes are kind of wonky, there's nothing we can do about it sadly XD

[isalin]

Great!

I'll probably try to make a version with a more cleaned up ui over the weekend.

Thanks for helping out with the testing!

[Aosys]

Ran a few other midis through, and noticed it's not handling some of the notes correctly. I think this is because it's attempting to play flat where it should be a sharp on the note below it (because of the wonky way NW bound things). Specifically, I can see it's playing down + 7 for F#, when it should be playing up + 6 instead. I'll have to see if other notes are similarly effected.

You can hear it happening pretty well with this midi: https://songs.bardmusicplayer.com/?dl=849277444607574016

[isalin]

Oh, I think I see where the problem might be!

I think I might've missed to have it let go of the arrow keys properly in some cases.

I was a bit too enthusiastic when I got it working, and just assumed the wonky notes were the strings sounding a bit odd.

I won't have time to check it out until tomorrow afternoon though.

Thanks for testing and reporting back!

[Aosys]

No problem, thanks again for doing this! Also, is it possible to do chords? I don't seem to be able to get multiple notes to play at once.

[isalin]

No, problem!

I'll look into chords.

In ffxiv, one of the limitations was that it couldn't process more than one key up/down event per frame. Which meant that you sometimes had long delays when trying to press multiple buttons at the same time. This made playing fast songs a lot worse back when they used a similar system to neverwinter, where you also had to hold down keys to move up and down octaves. It also made playing chords impossible.

It's why TBBard asks about your frame rate.

Neverwinter might not have the same limitations though!

[Aosys]

Ooh, interesting! I haven't had any issues with manually playing chords, though because of the system you can't have a chord with both naturals and accidentals (which is super annoying, but it is what it is...).

[Edit] Ran into another bit of weirdness. I tried running it on my other, slightly less powerful computer, and it's not performing properly. Windows 7 with a 970, versus my other Window 10 with a 1070. It's possibly lagging a bit and not playing all the octaves correctly, versus the same midi file running fine on my 1070 machine. Is this related to the frame rate detection, or is this possibly the same issue as FFXIV?

[isalin]

Interesting!

There does appear to be some sort of limit on how many notes can be played in a short interval.

I'm not sure if it's related to the framerate though.

If I lock my framerate to 60 fps, then one frame is ~16ms. But if I enter something like:

A
w2
A
w2
A
w2
A
w2
A
w2
A
w2
A
w2
A
w2

Then is should hit the "A" key every 2 milliseconds, 8 times. For me it consistently ends up processing it 4 times. If it was locked to the framerate, it should only process it 2 times.

Unfortunately I don't have a slower computer to test it on at the moment. But I would guess that it probably just polls the keys as often as it can.

If you try to play the above "song" using the new version below, does it also consistently process it 4 times?

---

The version I uploaded to you earlier still had code that checks and adds delays to key presses based on you framerate. That would probably prevent some chords from playing (if you wanted to press C+D, then it'd press C and wait a frame before pressing D).

In this version I changed it so that it doesn't add in wait time for a new frame. Which means that notes separated by "w0", or no wait at all will press at the same time.

I also tried to make sure that it properly lets go of the arrow keys. But that got me thinking! What if if it's that the octave switching happens to fast for it to process? So I made two versions. One with no delay for anything, and one that waits for a frame after switching steps!

v2.zip

v2_nostepdelay.zip

I feel like the first one with a delay for the arrowkeys sounds better!

Neverwinter doesn't appear to be bound by framerate the same way FFXIV was, so you should be able to set the framerate setting to something higher than your actual framerate. This delay indicator now only indicates how long it'll wait to play the next note after switching octaves.

Personally I set my fps to 150 while testing!

...Also, am I crazy or does down+4 not make any sound in-game?

[Aosys]

You're not crazy, down+4 is silent! I'll probably have to make a map of all the possible notes, since there's weird idiosyncrasies like that.

On my laptop (the more powerful machine) I get 4 notes for that sequence. On desktop (less powerful), I get 2, or sometimes even one... It should also be noted that on desktop, I have to run as administrator to get it to register with NW, and for whatever reason the arrow keys don't seem to work (TBbard can't switch octaves, nor can it do sharps/flats). It works just fine on laptop, no need to run as admin there.

I'll try experimenting with chords next!

[Edit] I went somewhere in-game with less lag on desktop, and was able to bring that sequence up to 3 notes, but no more. On laptop, I went to the same place and got 6 notes!

[Edit 2] They just pushed an update today; looks like it fixed the problem with down+4, which now has a note assigned. It looks like they also fixed the issue with down+7, which now plays F# correctly!

[isalin]

Ah, then it probably does just sample the keys as often as it can.

And great that they patched it!

But very weird that the arrow keys don't work. It presses the arrow keys the same way it presses the numbers. I'll try to look into it!

Edit: Oh, try enabling/disabling numlock and see if that makes the arrow keys work? There seems to be an issue where java prefers the numpad arrows for some reason.

[Aosys]

Tried with numlock off and on, and no dice with either. I wanted to say it's related to the fact that I'm running as administrator (which also has the side effect of not letting me drag and drop midi files into the program; I have to use the "open" function to actually get them loaded), but I just tried on my laptop and it's still working (minus the drag-and-drop issue). I couldn't say if it's an OS or JRE issue, I've never run into Windows 7 rejecting keypress detection before. My laptop is using Java SE Development Kit 13 (64-bit), while my desktop has Java 8 Update 291 (64-bit). Do I need the SE kit instead?

[isalin]

I'm not at my computer at the moment, so I can't check my specific version. But I feel like it shouldn't make a difference. Then again maybe there a bug with that's specific version or something.

I am running as admin on windows 10 though. And the arrow keys work for me.

Drag and drop is a bit annoying. If I remember correctly, you can't drag and drop from a non-admin application (explorer), to an admin application (TBBard). But you also can't send keypresses into some applications unless you're running as admin.

[Aosys]

What's infuriating is that the numbers are working just fine, it's just the arrow keys misbehaving. I also tried launching through the Arc launcher, as I don't think the game launches in admin mode when you do, but it still didn't work...

[isalin]

Yeah, it's very weird. I'd understand if it didn't send anything at all. But just not sending the arrow keys is very odd.

Especially since it's working on your laptop, and my computer.

Maybe try switching Java version after all?

[Aosys]

I just tried running TBbard, in Windows 7, on top of a browser keyboard tester and it's registering the left arrow key there (would probably register the other arrow keys for files using them), so I guess there's something weird with Neverwinter specifically. I can manually play all the notes properly, so I know it works in theory. I guess there's some kind of incompatibility with the application permissions of both apps? On laptop, I'm running NW through Steam (not as admin) and it's all smooth sailing from there.

[isalin]

I'm using the non-steam version through the arc launcher.

Just to double check, you haven't changed the key bindings in neverwinter or anything?

[Aosys]

Nope, all the keybinds I have are default, I double-checked to make sure. Tried through Arc launcher, numlock off, running TBbard as admin, and still only number presses work.

[isalin]

You're using the version with a delay after the arrow keys, right?

Try setting the FPS setting to something really low. Like 30, or even 10. Does that make a difference?

[Aosys]

Well whaddya know, it was a Java bug after all! I just installed the JDK instead of just the JRE, and it works a treat now! Odd, but I'll take it! Can set the FPS to 150, no problems!

[isalin]

Wonderful!

And a bit surprising! But great that it works!

[Aosys]

Regarding chords, I'm having some difficulties going from Musescore to TBbard. I did a tiny test trying to get two notes to play together. Here's how they appear in Musescore:

However, upon export to midi, they end up creating:

hF
w0
hA
w473
release
w26
hA
w473
release
w26
hA
w473
release
w26
hA
w473
release

In short, it seems only the top note gets detected almost all the time. Opening the file with other midi software plays all the notes just fine.

I don't know if this technically constitutes a TBbard issue though; I may have to go talk to some Musescore folks about it?

[isalin]

I'll look into it!

Now that you mention it, I have a vague memory of it maybe filtering out notes that happen at the same time while initially parsing the midi file. Since ffxiv had some limits on pressing keys simultaneously.

I don't suppose you could send me a simple file that just plays a chord or two for testing?

[Aosys]

Absolutely! Here's a very simple version of Chopsticks that should be suitable for testing:

https://drive.google.com/file/d/1S89yUEO1vkHzJIyIpAm5j0tVHUcgGYom/view?usp=sharing

[isalin]

Thanks! I'll check it out later today.

[isalin]

This version seems to parse chords correctly.

I think I might've just removed the "1 frame delay" from the playback last time. But there was still code that tried to take that delay into account when it first parsed the midi-file into notes.

v3.zip

[Aosys]

Hi there! Sorry I haven't replied in so long, I took a bit of a break from the game. However, I've hopped back in now that Bard has gone live, and so far TBBard works almost perfectly! However, I have noticed there are problems with some notes not firing correctly. In this case I had some high notes not registering, instead being played as mid notes; I have a feeling this would apply to certain sequences with low notes and/or sharps/flats (the sequence I was playing had quite a lot going on, and I've noticed some chugging the more complex the music is). Is this an FPS limitation on Neverwinter's end? I tried the same song on the preview server and encountered this issue too, but not with the same severity, things were broken at 150 Min FPS but got resolved by cranking it down to about 120. On live I think had to go as low as 90, and the rest of the song ended up sounding super stilted.

[isalin]

No worries. You have to enjoy the summer too, not just game! 😄

If it plays all the notes correctly when you lower the FPS, it's almost certainly related to the Neverwinter framerate/sampling rate.

From what I can tell, Neverwinter samples the current state of the keyboard as often as it can. How fast it happens depends on your computer. The problem is that you need to wait one "frame" to toggle one of the arrow keys on/off before a note will register correctly. So going up/down will always have to cause a small delay.

When you increase the FPS, you'll lower the delay. But if the delay gets too low, Neverwinter won't have time to register that the arrow key has been pressed before playing the note.

If you're playing some song that uses the arrow keys a lot, and quickly, it'll probably start sounding a bit stuttery and stilted. I'm not sure that can be fixed unless they add a way to bind the whole keyboard, without the arrowkey modifiers.