IDLE: work around shortcuts bug in Windows' IMEs and tk

[565fd44c-c138-4950-9753-e8a0e6f259e9]

BPO	31244
Nosy	@terryjreedy, @serhiy-storchaka, @mlouielu
Superseder	bpo-46052: IDLE: make Ctrl,Alt + IME non-ascii letter work on Windows

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = 'https://github.com/terryjreedy' closed_at = created_at = labels = ['expert-IDLE', 'type-bug', '3.7'] title = "IDLE: work around shortcuts bug in Windows' IMEs and tk" updated_at = user = 'https://bugs.python.org/ConstantineKetskalo' ``` bugs.python.org fields: ```python activity = actor = 'terry.reedy' assignee = 'terry.reedy' closed = True closed_date = closer = 'terry.reedy' components = ['IDLE'] creation = creator = 'Constantine Ketskalo' dependencies = [] files = [] hgrepos = [] issue_num = 31244 keywords = [] message_count = 15.0 messages = ['300623', '300624', '300667', '300682', '300685', '300687', '300693', '300694', '300716', '300737', '300745', '300746', '300855', '300934', '408416'] nosy_count = 4.0 nosy_names = ['terry.reedy', 'serhiy.storchaka', 'louielu', 'Constantine Ketskalo'] pr_nums = [] priority = 'normal' resolution = 'duplicate' stage = 'resolved' status = 'closed' superseder = '46052' type = 'behavior' url = 'https://bugs.python.org/issue31244' versions = ['Python 3.6', 'Python 3.7'] ```

[565fd44c-c138-4950-9753-e8a0e6f259e9]

Hi there. Thank you for Python, I love this programming language.

I've found a bug with shortcuts. I'm from Ukraine, so I'm working with English, Ukrainian and Russian languages. But any shortcuts, where alphabet is involved (like Ctrl+Z, Ctrl+F, Ctrl+V and so on) work only when English is turned on. This is quite inconvenient. It's possible to work with it, but still confusing, because you can select something in IDLE, copy it with shortcut Ctrl+C, then try to paste it to another place and then you find out that it wasn't copied, because active language is Ukrainian. So you need to switch to English and repeat everything again.

I'm currently working with Python 3.6.1, Windows 8.1 Embedded enterprize x64. Also I've noticed the same under Windows 7, Windows 10 in different versions of Python. I believe there where Python 2.7, 2.11, 3.4, 3.5. Not sure about all the versions exectly, but for sure shortcuts worked the same in all the versions I've ever tried to use.

I hope this was helpful and I'm sure many developers, that work not only with English would be really happy if you would fix this.

Thanks again for your time and effort. Best regards, Constantine.

[terryjreedy]

IDLE and Python/tkinter have nothing to do with the handling of keystrokes and their direct interpretation. The behavior you describe, the Ctrl key being disabled in Ukranian mode, strikes me as a bug in either Windows or the Input Method you use, or, less likely, the tcl tk GUI framework. Someone who uses the Japanese IME with IDLE thinks Ctrl continues to work right in Japanese mode, but will check and report to me.

IDLE offers menu entries for most shortcuts. Indeed, shortcuts are shortcuts relative to using the menu. Cut, copy, and paste are on the Edit menu. It should be accessible with Alt shortcuts. C, C, and P are also on the right-click context menu.

[565fd44c-c138-4950-9753-e8a0e6f259e9]

You must be kidding me? I know that it's possible to use it via menu. And my speed of software development will be 3 times slower. Great! If this was a bug in Windows then why the heck it works perfectly in any other program including Python when I switch language to English?

[9da18a29-a07d-4828-a801-24cc911246de]

Hi, so you are using Windows built-in IEM with Ukrainian input method. And in this situation, you cannot use Ctrl+Z, Ctrl+V ..etc. right?

[565fd44c-c138-4950-9753-e8a0e6f259e9]

Louie Lu, yes, I'm using Windows. But, sorry, what is IEM? I'm currently using native Python IDLE. Right click to .py file and "Edit with IDLE". However I'm starting to think about switching to Visual Studio, even though it's huge and slower... Yes, I'm using both latin and cyrillic characters, and if just finished typing some string in Ukrainian, combinations Ctrl + Z/X/C/V/F/G/H/S, also Alt + G: they just don't work. Only if I switch back to English.

[terryjreedy]

IDLE's involvement with cut, copy, and paste is to tell the tcl/tk GUI framework what keys to associate with the operations. The keys depend on the OS-specific defaults or the keyset chosen by the user. The actual operation on the Text widget is carried out by tk, in response to events it gets from the OS.

IME = Input Method Editor.  I just watched ^C and ^V work on Windows with Window's Japanese IME.  I properly closed this as an IDLE issue.  Serhiy, do you know anything about Windows' built-in Cyrillic IMEs and control key combinations?

[565fd44c-c138-4950-9753-e8a0e6f259e9]

Terry J. Reedy, Japaneese has no cyrillic characters, it's alphabet consists of hieroglyphs and it has nothing to do with Ukrainian even if it looks the same for you. "about Windows' built-in Cyrillic IMEs and control key combinations?" In depth - no. I'm just using them and I know that Python is the only program I've ever met that doesn't exept same combinations with Ukrainian or Russian language on. If you want to test what I'm talking about - just temporary add to your languages on your PC Ukrainian language, open Python IDLE and try any shortcuts while input language is Ukrainian. Cyrillic characters can have sometimes problems with encoding, because they are not included to default ASCII table, only to extended versionis. Could that be a reason? Or I just don't understand something? If not, please explain or give me a link or tip where can I find out what's goind on and who can fix it. Thanks.

[565fd44c-c138-4950-9753-e8a0e6f259e9]

*versionis = versions

[serhiy-storchaka]

I confirm the issue. Seems this is a particularity of Tk implementation on Windows (it looks as a bug from user's point of view). There is a workaround, but it is too complex. For the particular keyboard layout and for every registered shortcut you need to find the character emitted by pressing the corresponding key, find its code in the current ANSI codepage, find the name for this code in the keysym table [1], and register additional shortcut for this name. For example for making Ctrl-O working with Ukrainian encoding you need to add \<Control-Key-ugrave> open-window-from-file action. Cyrillic letter "щ" is emitted when the key "o" is pressed in Ukrainian layout, it has the code 249 in CP-1251, which corresponds the name ugrave in the keysym table. All this is too complex for IDLE. I think that we should close this issue as "Won't fix" and hope that this will be fixed automatically in future versions of Tk.

Firefox starved from similar issue for many years. It was fixed in recent versions. But Firefox is much more complex program than IDLE.

[1] https://www.tcl.tk/man/tcl/TkCmd/keysyms.htm

[terryjreedy]

Constantine: the point of checking the behavior of the Japanese IME was to determine whether there is a problem with all IMEs or just some. Since Ctrl + the 2nd key from the left on the bottom row (marked 'X') invokes 'Delete' in both American and Japanese entry modes, the behavior you want should be possible.

I do not know where the bug is, and do not care, other than knowing that it is not in IDLE (or Python). I do care that anyone in the world, like you, with sufficient knowledge to use Python, be able to also use IDLE if they chose. Even if switching keyboards with alt-shift is a fast, habituated, motion, it is clearly a nuisance in this context So, even though IDLE is not Firefox, I am willing to consider somehow implementing what Serhiy described and supporting either adding workaround keyset(s) to IDLE or make it easier for users to create their own custom keysets that work.

When I mentioned the menus, I assume that the alt menu shortcuts work, so alt-e, alt-c, for instance, would copy a selection. When we tried this with the Japanese keyboard, the alt shortcuts work for the top menu (F_ile, E_dit, S_hell, etc) but not for the drop-down submenus. After alt-e displays the edit menu, alt-c, etc, have no apparent effect. Another bug, and one which I doubt we can work around without language-specific menus. (This is a separate issue.)

Step 1 for this issue: what is the scope of the problem. I presume it is not unique to Ukranian. Does it affect all Cyrillic IMEs? If so, do different cyrillic keyboards have the the alphabetic chars in different places, so that a separate workaround is needed for every IME, even for letter chars?

[serhiy-storchaka]

Yes, it affects all Cyrillic layouts, as well as Greek and other alphabetical systems. It also affects layouts for Western-European languages with keys for accented characters. Different Cyrillic keyboards have different characters on the same key, but I don't know if they have different keys for the same character. Fortunately all Cyrillic languages use the same ANSI encoding. But I think there are conflicts between Cyrillic and non-Cyrillic layouts. Different keys on different keyboards can produce the same code (which means different characters in different codepages).

[565fd44c-c138-4950-9753-e8a0e6f259e9]

Bug is not in Python? Then where? Windows?

• I assume that the alt menu shortcuts work...
• NO! And I've written that before as well! Alt + combinations don't work neither with

cyrillic!

• Does it affect all Cyrillic IMEs?
• Yes, I've written that before already - shortcuts don't work neither with Ukrainian, nor

with Russian

• If so, do different cyrillic keyboards have the the alphabetic chars in different

places, so that a separate workaround is needed for every IME, even for letter chars?

• Yes and no. Most letters in different cyrillic alphabets are the same, so they have the

same place on the keyboard(as well as the same encoding), but some of them are different,

so in different IMEs same key gives different letter. For instance English 's' in Ukrainian is 'і' and in Russian is 'ы'. But English 'c' is in

both Ukrainian and Russian 'с' (looks the same, has different encoding then English, but

has same encoding for Ukrainian and Russian).

[terryjreedy]

I obtained more info on the Japanese IME. It has 2 modes of operation: 'romanji' and 'kana'. In romanji mode, one types the ascii transliteration of the syllabic chars. When one types type the syllable vowel , the program replaces the sequence of 1 to 3 ascii chars with a Japanese char. So while typing 'akitsu', the 'a', 'ki', and 'tsu' would become 3 chars, with the 'k' and 'ts' momentarily visible. Modifier combinations naturally work since the keyboard is being interpreted as outputting ascii chars. In kana mode, japanese chars appear directly, but Ctrl must, in effect, temporarily revert the keys to their ascii interpretations.

A Chinese keyboard with a pinyan (romanization) input mode might do the same. But non-English keyboards having an ascii mode is likely exceptional. Given this and the two answers above, I conclude that delivering IDLE with a 'complete' set of IME-compatible keysets is an impossible problem.

I definitely want to document the problem.

I will stay open to the possibility of a 'customizer' that would prompt a user to hit all the character keys in a defined order and then augment an existing keyset using the method described in msg300716. The table at https://www.tcl.tk/man/tcl/TkCmd/keysyms.htm suggests that this could potentially cover accented latin, cyrillic, greek, hebrew, arabic, and japanese keyboards.

[565fd44c-c138-4950-9753-e8a0e6f259e9]

Impossible... Yea, right... Will documenting it help to solve it? If that's impossible, then how do other programs deal with it? What does make Python so special? I still don't see anything impossible. If other programs can do it but Python IDLE can't do it for now, then it's called either "hard" or "I don't want to do it", but not "impossible".

First of all I would try to find out how does cyrillic work with other programs(sorry, I don't know that). Probably their approach to this situation could be just copied to Python IDLE behaviour.

Japanese wan't help you with cyrillic. This ticket is about cyrillic!!!

ascii is not the only encoding, that exists. Could it be as possible solution to use unicode incoding with Python when it comes down to hotkeys?

If there is something that makes Japaneese and cyrillic languages incompatible with each other to use hotkeys, well, then you could just add option to choose which language is preferable to user (either in installed program, or during installatioin).

Windows command prompt doesn't support cyrillic by default, but there is a code in C, that makes it work with cyrillic input and output. Python was written on C, so I'm sure it could be helpful even if it doesn't work directly. Here is this code for C/C++: SetConsoleCP(1251); SetConsoleOutputCP(1251); setlocale(0, "Ukrainian");

[terryjreedy]

What makes IDLE different from other desktop apps is that is it written in Python, uses the tkinter wrapping of cross-platform tcl/tk, and allows users to customize nearly all hotkey shortcuts. But tk only allows Ascii chars, with modifiers, for hotkeys. This issue would be much easier if IDLE had a fixed set of bindings, or even a fixed set for each major platform.

bpo-46052 is a duplicate of this. In experiments reported there, using the Win10 Russian IME, I determined that the event.char for a Ctrl + letter-key combination is the corresponding Ascii control character, even when switched to Russian. The difference is that event.keysym is '??' instead of an ascii letter and event.keysym_num is the unicode ordinal of the russian letter instead of the ascii letter. So ('c', 99) becomes ('??', 1089). I propose on bpo-46052 to solve these issues by undoing this change and generating the event that would have happened in ENG mode.

[chenyulue]

Same issue comfirmed with Chinese-inputting keyboard layout even in python 3.10. For example, when I press key p, IDLE opens the python docs, and pressing r opens a search dialog.

Only when the keyboard layout is selected as English do the shortcuts in IDLE work well.

[terryjreedy]

@chenyulue Copied comment to open duplicate issue listed above.