search

TerryCavanagh / VVVVVV

The source code to VVVVVV! http://thelettervsixtim.es/
Other
7k stars 559 forks source link

Enable SDL_HINT_IME_SHOW_UI to make typing CJK not guesswork #1041

Closed Daaaav closed 9 months ago

Daaaav commented 1 year ago

Changes:

For some reason, the default behavior of SDL and/or Windows(?) (I only tested this on Windows) seems to result in the fact that if any SDL app doesn't account for it, there is no way for Japanese and Chinese speakers to know what they're typing in.

How IMEs are supposed to work is that you can type words as sort of WIP versions, and then select out of a list of candidates what the final result should be. The app may display the WIP text and tell the IME where the text field is so that the IME's menu can be displayed around it. But if the app doesn't say where the text field is, then the candidate list can also be displayed at the corner of the screen, which is done in Minecraft.

By default, however, SDL apps don't get a candidate list at all, which means you're basically flying blind as to what you're typing in, and you would have to basically open notepad and copy-paste everything from there - unless I'm missing something.

This commit sets the SDL_HINT_IME_SHOW_UI hint (added in SDL 2.0.18 apparently), so that the candidate list is at least shown in the corner. We can probably deal with positioning and uncommitted text later.

Demo video with this hint enabled:

https://github.com/TerryCavanagh/VVVVVV/assets/44736680/2b13b5d9-1254-4972-8ae3-33dbccce4813

With this hint disabled, functionality is the same (you can arrow key through the candidates) but the entire interface in the bottom right is not there.

Legal Stuff:

By submitting this pull request, I confirm that...

flibitijibibo commented 1 year ago

I'd have to double check but I believe the candidate list also shows up after setting SDL_SetTextInputRect, which gives the window a place to sit near the input region. Do we already call that somewhere?

KabanFriends commented 1 year ago

Unsure if it's related, but there is another text input box that shows what you are actually typing (marked red in the image) Using IME you can type multiple words at same time without sending it to the application
This input box is usually merged with app's own input boxes but I have seen some games that do not have a dedicated system for it and uses this fallback? input box instead

image

flibitijibibo commented 1 year ago

Does SDL_TEXTEDITING give us this information as well? It would take some more work to make it work with the text buffer but that should help make this more integrated with the in-game text UI.

InfoTeddy commented 1 year ago

Do we already call that somewhere?

I don't think so.

Daaaav commented 1 year ago

I'd have to double check but I believe the candidate list also shows up after setting SDL_SetTextInputRect

According to https://wiki.libsdl.org/SDL2/SDL_SetTextInputRect, it doesn't give any feedback without setting SDL_HINT_IME_SHOW_UI.

Unsure if it's related, but there is another text input box that shows what you are actually typing (marked red in the image) [...] I have seen some games that do not have a dedicated system for it and uses this fallback? input box instead

That would be nice in the meantime, but I haven't yet found a way to get that to show up.

Does SDL_TEXTEDITING give us this information as well? It would take some more work to make it work with the text buffer but that should help make this more integrated with the in-game text UI.

Showing the info from SDL_TEXTEDITING is the ideal way of working yeah, but probably takes a bit more work; everywhere there's a flashing cursor in the game, we need to display the text and two distinct visual styles of underlining/color, while accounting for centering. But looking it up now there aren't that many text inputs in the game, so maybe it's not too bad to change them all...

Daaaav commented 1 year ago

Update: it looks like SDL_TEXTEDITING is a bit broken and/or doesn't give enough information.

Apart from the not-yet-committed text, there's two properties, start and length. In all my testing, length is always 0, no matter what. start isn't too useful either when switching between different words in the same phrase, because it doesn't change when switching between the words. Am I missing anything?

This is how it looks in Wordpad (with audible keypresses for slightly better context):

https://github.com/TerryCavanagh/VVVVVV/assets/44736680/80a1e551-aa68-47d1-9a4e-33bd9e0f701c

This is how the same keypresses look with SDL_TEXTEDITING:

https://github.com/TerryCavanagh/VVVVVV/assets/44736680/bda33b64-95e2-4348-a482-656f6a50b05f

So as it looks like now, we can show the entire WIP text, but not which substring is "active".

Daaaav commented 12 months ago

@InfoTeddy could you try to confirm if you get different (maybe more logical) values for start and length in SDL_TEXTEDITING with your IMEs? I think we should just implement it how the API is probably intended; even if the underline won't be correct with Windows' default IME (or maybe even all IMEs), and even if setting the SHOW_UI hint doesn't cause all IMEs to show. Those are probably things for SDL to solve.

Daaaav commented 11 months ago

For reference, just reposting this question from Discord:

hmm is there a way to calculate coordinates from the game's native resolution (320x240) to where that pixel will appear in the final window? (so depending on stretch mode, scaling, how big the letterboxes/pillarboxes are, etc). i'm trying to use SDL_SetTextInputRect to make the IME candidate list appear somewhere better and it works but it applies it as if the entire game is 320x240 in the top left of the window. all SDL functions that seem useful, including SDL_RenderLogicalToWindow or SDL_RenderGetScale, always just give a 320x240 result. so idk if i'm looking at the wrong place or if this will be a pain of having to manually reproduce it by looking at the real window size and every scale/stretch option we have and predicting what the result will be

In short, I'm kinda stuck at positioning the IME window correctly because SDL_SetTextInputRect seems to ignore all graphics transformations, as well as all other functions specifically for getting information about the graphics transformations. Right now I'm thinking maybe the graphics state is only set in a small part of the code for drawing one frame and then reset again so all the rendering can pretend the window is 320x240?