[SDL2]: Some keyboard keys acting up after (to window) mode switch

[raziel-]

SDL2.0.9RC2 - MiniGL2.23

I'm afraid this will be a hard one to test, as i only found it to happen in the Grim Fandango full game, it doesn't happen in the demo.

If i switch the game from screen to window (ALT+ENTER) the cursor keys "left" and "right" (cursor keys and numpad) won't react anymore, while "up" and "down" (also cursor keys and numpad) still work fine.

On switching to window, it "seems" to be activated (at least that is what i "see", the window border is blue instead of grey and the mouse cursor is hidden and won't come up when i move the mouse around, not even when moving it outside the window border!...unless i click somewhere outside the window area)

Now the strange part: If i click once into the residualvm window area, ALL of the cursor keys start working again. So i'm guessing that the window (after the mode switch) isn't correctly or fully activated and as such lose some of the keyboard keys to be interpreted as pressed (which would sound logical to me, because the SDL window wouldn't be "active" at that time - why the other keys still work though, is beyond me).

Clicking into the window area to make it "active" would be a fine little workaround, if there wouldn't be the fact that fullscreen is also affected after switching the mode back to it. Because, if i switch back to fullscreen (regardless if i applied the workaround mentioned above) the cursor keys ("left" and "right") will again be lost and won't react. The worst thing now is, that i cannot use my workaround in fullscreen as clicking anywhere into the play area doesn't trigger an "activation", so i can't fix it and thus i'm lost with some of the keys keys not reacting and the game not being playable anymore. (Unless i switch back to window, apply the workaround and play the game in window mode)

I'm guessing that the mode switch(es) doing something they shouldn't...

edit: Revised the original bug item, since it only affects the SDL2 build

[capehill]

To me it sounds like a game (engine) issue where engine switches between some mode internally. But, we can debug keys that SDL receives from AmigaOS, and probably, after enough coffee and time, we can find the piece of code in ResidualVM that is responsible for handling (cursor) keys and add some debug, too.

[raziel-]

I got feedback from another user that runs residualvm under linux with SDL and he doesn't get this behaviour at least. Not that it helps the cause :-/

[raziel-]

Thank you for always being available and sorry for taking up so much of your time

[capehill]

You could try this debug lib: http://capehill.kapsi.fi/sdl2/sdl2_keyboard_debug.lha

Use sashimi or serial cable to see the key presses.

[capehill]

Any news regarding keyboard debug?

[raziel-]

Funny, i thought i already did the tests and even posted the results...must have been a dream.I'll get on it asap (11th of may earliest).Thanks for the heads up. null

[raziel-]

Ah, i remember now why I haven't testet it yet...because if the compiler error I'm getting...see the other bug item. null

[raziel-]

I hate Odyssey

[raziel-]

ResidualVM - SDL1 - behaviour is gone. ResidualVM - SDL2 - behaviour still there.

Test with the keybord debug is next, bear wih me

[raziel-]

OK, the debug shows that the key presses are being recognized, but the character still doesn´t move. GitHub.txt

[capehill]

It should be checked on ResidualVM side whether those keys are received and what happens then.

[raziel-]

I sent a mail to one of the ResidualVM maintainers, as this is beyond my capabilities.

[capehill]

You could try to locate 2 places in ResidualVM codebase: 1) place where SDL events are handled, and 2) game-specific event loop

And the add some debug logging.

[raziel-]

:-) you forgot who you talking to..."beyond my capabilities" ;-)

[capehill]

You can search text with "SDL_event" and write log lines, so ... :)

[raziel-]

Ok :-)I will try, but be prepared for more questions ;-) null

[raziel-]

SDL2 is affected, SDL1 is not.

I did check some of the SDL_events and i can make out differences when using SDL2, but thats about it. I don´t know enough about coding to try and fix anything.

Sorry

[capehill]

To debug Grim events specifically, one might start somewhere here https://github.com/residualvm/residualvm/blob/master/engines/grim/grim.cpp#L853

and for example print key numbers (event.kbd.keycode) to console. I guess something like this might work:

warning("keycode %d", event.kbd.keycode);

I didn't peek inside handleControls() function yet - might be even better place.

[raziel-]

With that example i can at least rule out that the game has some issues with the keyboard typing.

WARNING: keycode 275! WARNING: keycode 276!

275 is cursor right, 276 is cursor left. I get both when pressing them, but no movement happens when i start typing them after i have switched to window.

As i wrote earlier, as a workaround i need to "deactivate" the window (click anywhere else so that the window border becomes grey) and click into the window again to make it "active" again. Only then will the character move on pressing the left and right cursor keys.

It seems to have to do with "activating" the window, but i'm not sure why it would only affect those two kleys. A workaround would probably be to "manually" (inside SDL2) activate the window once it has been created (if that is even possible, i'm not really sure what i'm talking about) :-)

[raziel-]

Also i don't really understand why SDL1 works...is SDL1 doing so much different with keyboard handling?

[raziel-]

I'm wondering if SDL1 window creation is doing something different which either SDL2 can't or is broken?

[capehill]

Try putting a debug print also here: https://github.com/residualvm/residualvm/blob/master/engines/grim/grim_controls.cpp#L306

Log both operation and key.keycode.

After that events are passed to the script and if it's proven that those problematic key events arrive correctly here, the problem must be even deeper.

[raziel-]

Here is the log: (This is when switched to window. 1) Pressing two times cursor right 2) Pressing two times cursor left Both with no movement of the character 3) Pressing two times cursor up 4) Pressing two times cursor doen Both with character movement

WARNING: gim_controls.cpp: operation 1 - key.keycode 275! WARNING: keycode 275! WARNING: gim_controls.cpp: operation 2 - key.keycode 275! WARNING: keycode 275! WARNING: gim_controls.cpp: operation 1 - key.keycode 275! WARNING: keycode 275! WARNING: gim_controls.cpp: operation 2 - key.keycode 275! WARNING: keycode 275! WARNING: gim_controls.cpp: operation 1 - key.keycode 276! WARNING: keycode 276! WARNING: gim_controls.cpp: operation 2 - key.keycode 276! WARNING: keycode 276! WARNING: gim_controls.cpp: operation 1 - key.keycode 276! WARNING: keycode 276! WARNING: gim_controls.cpp: operation 2 - key.keycode 276! WARNING: keycode 276! WARNING: gim_controls.cpp: operation 1 - key.keycode 273! WARNING: keycode 273! WARNING: gim_controls.cpp: operation 2 - key.keycode 273! WARNING: keycode 273! WARNING: gim_controls.cpp: operation 1 - key.keycode 273! WARNING: keycode 273! WARNING: gim_controls.cpp: operation 2 - key.keycode 273! WARNING: keycode 273! WARNING: gim_controls.cpp: operation 1 - key.keycode 274! WARNING: keycode 274! WARNING: gim_controls.cpp: operation 2 - key.keycode 274! WARNING: keycode 274! WARNING: gim_controls.cpp: operation 1 - key.keycode 274! WARNING: keycode 274! WARNING: gim_controls.cpp: operation 2 - key.keycode 274! WARNING: keycode 274! WARNING: gim_controls.cpp: operation 2 - key.keycode 13! WARNING: keycode 13! WARNING: gim_controls.cpp: operation 2 - key.keycode 308! WARNING: keycode 308!

[raziel-]

There is one more oddity to this behaviour.

If i switch to window, open the options menu with F1, navigate to "option" and level the music volume it works with left and right cursor. There it is picked up perfectly. But going back to the game screen with ESC two times, the character will still not move.

[raziel-]

I had a little more time to test and found that from the three games residualvm supports the two which feature keyboard controls are affected. It's not limited to Grim Fandango, which even more points to something outside residualvm.

I'm suspecting something in the keyboard handling of SDL2.

Then again, i've encountered something similar in ScummVM, when keyboard fighting (in Indiana Jones 3 and 4) broke, due to a fix they introduced for another platform.

Need to check if i find something similar in residualvm.

[raziel-]

btw: The other game that is affected is Monkey Island 4, albeit in this game the cursor keys "up" and "down" refuse to move the character when in window mode.

And for the sake of completeness, not only the cursor keys are affexted, but most of the "normal" keys aswell. e.g. "e","s", "u". While "i", "ESC" and the number keys above the keyboard work fine.

Really strange one

[raziel-]

@capehill

Ok, since Monkey Island 4 is also affected and after testing the demo of it, i´m glad to have a test case at last, that i can share.

I´ll send it to you privately. The demo files and the latest residualvm debug build together with the correct .ini file, so you can delve right in.

1) Start residualvm 2) Start Monkey Island 4 demo You´ll instantly see the other problem i´m getting (white screen on videos), disregard that for now (maybe later?) ;-) 3) Press ESC 2 times o get control over Guybrush 4) Press the cursor keys in fullscreen to see how he behaves 5) Press LALT+ENTER ti switch to window mode 6) Press the cursor keys again, only this time "up" and "down" will not move Guybrush

[capehill]

I wasn't able to start your build because it seems that was linked using z.library 53.6 while I have 53.5 or something. So I tried to update my build but have some trouble: a DSI pointing to bitmap.cpp - any ideas?

Meanwhile I wasn't able to reproduce the keyboard problem on Linux with Monkey4 demo (I have also the full game just in case).

By the way, white screen issue in MiniGL is most likely some bad texture parameter (size, format) and you may want to check the error logs in case GL error was logged.

[raziel-]

That is a problem since i don't remember updating z.library anytime. I guess it was done through amiupdate? I have the update lib installed but am unfortunately not on my setup for next two weeks. So sending it to you is impossible right now.

Wrt to the linux test, thank you, that can mean that is really a genuine amigaos4 bug.

And I'll check for gl errors in residualvm once I'm back as well.

Thanks for the heads up.

[raziel-]

http://www.amigans.net/modules/xforum/viewtopic.php?post_id=115677#forumpost115677 This is the reason for the error you are getting...stupid :-(

I'll send you a workaround

[capehill]

Thanks, I got your binary running. (MiniGL rendering is broken (my MiniGL? or drivers?), the polygons are a mess, but SW rendering works better, ignoring the endianness)

I was actually able to reproduce the behaviour (UP/DOWN non-functional), but not every time. It seems as if ALT key state would stick sometimes. This could have something todo with the earlier issue where special keys were not properly updated during window re-creation. Maybe "key modifer sync" somehow badly interleaves with other SDL events and gets engine into an "ALT" state. When pressing ALT, it's possible to turn Guybrush but not walk forward or backward.

Toggling ALT key is a workaround - can you confirm?

It would also explain why SDL1 behaves differently. I don't know why only certain game versions suffer from it.

[raziel-]

wrt MiniGL rendering: Probably because i still use a (not released) MiniGL beta which fixes z-ordering and adds some stuff by Daytona)

Thank god you can reproduce...i began to think i have a ghost in the machine flipping the finger towards me once i start the x1000

Toggling ALT onnce winndow is up is curing the problem, yes Then i remembered i had a similiar bug (Modifier key not released) in the sourceforge issue list which you fixed https://sourceforge.net/p/sdl2-amigaos4/tickets/38/ Seems like ALT is also affected (or replaced CTRL in that matter)?

[raziel-]

@capehill

quote: By the way, white screen issue in MiniGL is most likely some bad texture parameter (size, format) and you may want to check the error logs in case GL error was logged.

How would i do that? Your GLError logging example from ScummVM doesn't work with residualvm due to a different code base.

Thank you

[capehill]

What is the exact problem you are encountering with GLError logging? I searched briefly Residual code on Github and found only glGetError calls in Android backend so probably something has to be added.

[raziel-]

Yes, i know i have to add them myself, but i don't know where. There is so much OpenGL stuff that i'm lost finding the correct spot to put the logger.

Sorry :-(

[capehill]

I would start near buffer swap, https://github.com/residualvm/residualvm/blob/6e02a103abea8e43784284f9a3181c1d90c2092e/backends/graphics/openglsdl/openglsdl-graphics.cpp#L538

This is the important function which is called every frame. Code doesn't have to be inside #ifdef block, and it shouldn't matter much is it before or after swap. Also if you are in contact with Residual devs, please ask them about possibility to add GL error checking (or how to activate existing mechanism if there is one).

[raziel-]

Thank you, continued here: http://www.amigans.net/modules/xforum/viewtopic.php?post_id=115729#forumpost115729

[raziel-]

Ok, so i added this

ifdef amigaos4

// GLError checking GLenum err; // GLError end

endif

void OpenGLSdlGraphicsManager::updateScreen() { if (_frameBuffer) { _frameBuffer->detach(); glViewport(0, 0, _overlayScreen->getWidth(), _overlayScreen->getHeight()); glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT); _surfaceRenderer->prepareState(); drawSideTextures(); _surfaceRenderer->render(_frameBuffer, _gameRect); _surfaceRenderer->restorePreviousState(); }

if (_overlayVisible) {
    _overlayScreen->update();

    if (_overlayBackground) {
        _overlayBackground->update();
    }

    drawOverlay();
}

if SDL_VERSION_ATLEAST(2, 0, 0)

SDL_GL_SwapWindow(_window->getSDLWindow());

else

SDL_GL_SwapBuffers();

endif

// GLError checking
while ((err = glGetError()) != GL_NO_ERROR) {
    warning("openglsdl-Graphics Swap (GL error): %d", err);
}
// GLError end

if (_frameBuffer) {
    _frameBuffer->attach();
}

}

to https://github.com/residualvm/residualvm/blob/master/backends/graphics/openglsdl/openglsdl-graphics.cpp#L516

and i don't get any GLerrors.

[kas1e]

I have some more info about that "alt key" issue when switching video modes, will copy it from amigans.net thread just so it not forgotten and will be there:

An Issue happens only with SDL2, not with SDL1.

So, I do check the SDL1 version and it seems that when you hit "alt+enter", then SDL1 hit another "alt" ourselves. I can see it in file manager used in DOSBox: when a single alt is hitting, it makes some visual action (blinking cursor over the filename), so I can visually see what happens and when with "alt".

I.e. you press alt+enter, it switches mode, and then SDL1 hit and realize "alt". Now, if after that you hit again "alt+enter", then it switches to another mode, but this time "alt" didn't hit again (seems it take second time's "alt" hit from alt+enter combo in account). But if you do any action after switching mode (i.e. when SDL1 hit "alt" after), and then "alt+enter", then SDL1 always will auto-press/realize ALT.

In other words, it seems in SDL1 we have a logical workaround for, or just handling loop done like it has done to mimic necessary behavior.

[capehill]

@raziel- Please retest with new release.

[raziel-]

Sigh

Yes, the issue is fixed, but introduced a new one.

Now the SDL2 build is as slow as melasse. OpenGL renderer peaks at 2 FPS (in-game) and 5 FPS (video sequences) while the same source base built with SDL1 is acting just as fast as it should (FPS 10 in-game, FPS 30 video sequences)

[capehill]

Sigh

Yes, the issue is fixed, but introduced a new one.

Now the SDL2 build is as slow as melasse. OpenGL renderer peaks at 2 FPS (in-game) and 5 FPS (video sequences) while the same source base built with SDL1 is acting just as fast as it should (FPS 10 in-game, FPS 30 video sequences)

OpenGL (MiniGL) context code hasn't been changed in 11 months. Please check your serial if there are some left-over debug prints causing slowdown...how do you measure FPS?

[raziel-]

Nothing on serial, no debug output in it. Clean source build

"show_fps=true" in residualvm.ini make Grim Fandango display it's FPS in the top left corner

[kas1e]

@Raziel And a version of SDL2 from 07 Oct 2019 (or so, i.e. when you start firstly check alt+enter bug) works fine and give good fps?

[raziel-]

@kas1e

Exactly. .9 is working perfectly fine (in-game 10 FPS, video sequences 30 FPS .11 is dog slow (both in the launcher and in the game)

The whole OpenGL part seems to be slowed down, Something inbetween versions have snuck in slowed openGL down.

(I can't do a cross check with ScummVM since ScummVM doesn't really use OpenGL)

I am well aware that this is beyond this bug item (i don't know what is causing the slowdown, probably something else than the keyboard fix), so i'm happy to open another one.

[kas1e]

@raziel- I mean you restest right now on the same config (with same minigl, etc, etc), just 2 different binaries : on e compiled with 2.0.9 all fine, and one compiled with 2.0.11 all slow? (i ask so we can avoid any misconfiguration, or different settings , or anything like you have 2 different dirs with different setup files, and can by mistake change something in one of them).

So to have good test case need to have just 2 binaries and run them both from the same place on the same setup, from the same directory (so it will take same config files) , one after another to see if there is indeed that issue.

[raziel-]

What you ask i did. Thats what i wrote in the earlier post.

Tested exactly as you asked and yes 2.0.9 is fine, while 2.0.11 is slow

[capehill]

Relinked my ResidualVM binary. Luckily slowness can be seen already in the launcher. Running tequila I can see SDL timer thread is taking 90% of the CPU so it points some issue with native threads. I will create a new ticket.

[raziel-]

Thank you.

Can we keep this ticket open until i was able to do a more thourough test? I did a quick one, but it is so unbearable slow that i'm not sure i'll be able to test any possible circumstance. Just so that i won't have to reopen it later...

[capehill]

Can we keep this ticket open until i was able to do a more thourough test? I did a quick one, but it is so unbearable slow that i'm not sure i'll be able to test any possible circumstance.

Yes, I would also prefer to keep this open until it's finally solved.