springcomp / optimized-azerty-win

AFNOR Compliant AZERTY Keyboard Layout driver for Windows
https://springcomp.github.io/optimized-azerty-win/
Creative Commons Attribution 4.0 International
52 stars 4 forks source link

Windows built-in support #114

Closed Slion closed 1 year ago

Slion commented 1 year ago

According to kbdlayout.info old AZERTY is now legacy and new one is standard, though legacy is still the default. Yet on Windows 11 I still can't find built-in support for the new layout.

springcomp commented 1 year ago

Thanks for the information. That is very interesting. Can you check your Windows version?

I’d like to investigate this and document that Windows now supports AZERTY-NF natively. Although, the version in this repository is still more powerful 😉.

Slion commented 1 year ago

I checked on the latest Windows 11 and the guy at kbdlayout.info mentioned his stuff is from Windows 10.

springcomp commented 1 year ago

The kbdlayout mentions version 10.0.25272.1000 which is very very recent.

I have not seen this version in the wild yet.

miloush commented 1 year ago

This should be available in public Windows Insider builds. Thanks for the "Windows 10" note, I should update that. :)

Slion commented 1 year ago

@miloush is the guy running kbdlayout.info. Thanks again for helping us out with this.

Would it be possible to generate such XML file for the layout that @springcomp is providing? @springcomp does your software include such a keyboard DLL that @miloush is using to generate that XML file?

miloush commented 1 year ago

The DLL file is the result of building the layout. I believe I can generate XML file from the KLC files too. Have you tried them here? http://kbdlayout.info/viewer might be able to give you the XML file from KLC.

springcomp commented 1 year ago

Analyzing the differences one can observe that:

I’m not sure if there is a correct way. Nor do I understand the full impacts.

It appears that the implementation in this repo is indeed incorrect. Fixed by #122.

Although this character does not have a defined mandated position in the standard.

I believe the Windows implementation to actually be incorrectly interpreting the standard. Edit: it appears the standard has conflicting requirements in different parts of the specification. Fixed by #121.

Fixed by 119.

Although the character is already available using AltGr+¤, Space in both implementations.

For this reason, both implementations chose:

  • To associate the Eu key to the U+017F LATIN LONG LETTER S.
  • To use the following simpler key combinations instead:
    • AltGr+i, AltGr+Eu to output U+1E9B LATIN LONG LETTER SMALL S WITH DOT ABOVE.
    • AltGr+k, AltGr+Eu to output U+1E9C LATIN LONG LETTER SMALL S WITH SHORT STROKE OVERLAY.

This post will be updated as further analysis is performed and shared here

Slion commented 1 year ago

AltGr+M does infinite symbol on the Windows layout but yours does not have that it seems. Is that missing or is it not in the specs. I rather like it. Can we have it? 😁 It is in the specs.

springcomp commented 1 year ago

@Slion the layout is definitely designed to output using AltGr+m (lowercase m). I use this layout daily on at least three computers and happen to use this symbol a lot as well. It definitely works there.

Please, can you double-check and tell me which version of the layout you have?

Maybe try Ctrl+Alt+m which should be equivalent to using AltGr.

Slion commented 1 year ago

v1.7.0.0 on Windows 11. Tried both the digits and the other layout. That's odd.

springcomp commented 1 year ago

@Slion Would you be able to run the compliance tests on your machine ?

Slion commented 1 year ago

I did not get around running those tests yet. However I was just trying it out on a Windows 10 machine and it works great there. I'm guessing this is a Windows 11 issue. It could also be that some other program is hijacking that key combination though. Probably the most likely issue I guess.

jbrianceau commented 1 year ago

The kbdlayout mentions version 10.0.25272.1000 which is very very recent.

I have not seen this version in the wild yet.

It has been officially released in Windows 11 Insider Preview Build 25247 that was announced here, in the [Input] section:

We are including two new keyboard layouts in this flight. These keyboards implement the two new French keyboard layout standards (AZERTY and BÉPO). The new layouts are designed to allow the user to type all the required characters of the French language. They also include support for all the Latin-based languages of the European Union as well as Greek letters and a large variety of scientific, mathematical, and financial symbols. To enable one of these layouts, go to Settings > Time & language > Language & region and select Language options under the ellipsis for the language you would like to use this keyboard with. On the Options page, select “Add a keyboard” and look for the new keyboard layouts “French (Standard, AZERTY)” or “French (Standard, BÉPO)”. The previous AZERTY layout now displays as (Legacy, AZERTY).

Slion commented 1 year ago

That's fantastic news! Hopefully Android will follow and hardware manufacturers too. It took them only 4 years since the publication to implement it. I'm impressed 😁

springcomp commented 1 year ago

@jbrianceau that’s indeed great news.

For the sake of my sanity, @jbrianceau could you please 🙏 double-check this issue above with regards to greater-than and less-than signs ?

As a hobbyist I could never actually check the official standard document – although I have sources that did, indeed, check the official document. My understanding is that those two characters are incorrect in the Windows layout 🤔.

jbrianceau commented 1 year ago

I have access to a printed version of the NF Z 71-300 French norm from April 2019. Here's what I can tell:

Code Unicode Glyphe Nom
U+0023 # Croisillon
... ... ...
U+2264 Inférieur ou égal
U+2265 Supérieur ou égal
... ... ...

However, when looking at the detailed implementation of AZERTY layout later in the specs:

Touche Goupe 1 / Niveau 1 Groupe 1 / Niveau 2 Groupe 2 / Niveau 1 Groupe 2 / Niveau 2
... ... ... ... ...
B00 U+003C < U+003E > U+2A7D ⩽ U+2A7E ⩾
... ... ... ... ...

Same for the BÉPO layout:

Touche Goupe 1 / Niveau 1 Groupe 1 / Niveau 2 Groupe 2 / Niveau 1 Groupe 2 / Niveau 2
... ... ... ... ...
E04 U+0028 ( U+0034 4 U+005B [ U+2A7D ⩽
E05 U+0029 ) U+0035 5 U+005D ] U+2A7E ⩾
... ... ... ... ...
Touche Symbole gravé Goupe 1 / Niveau 1 Groupe 1 / Niveau 2 Groupe 2 / Niveau 1 Groupe 2 / Niveau 2
E04 ( U+2264 ≤
E05 ) U+2265 ≥
... ... ... ... ... ...

So there's definitely a discrepancy here between the table 8 of the beginning and the rest of the document, but my understanding is that the spec intent is ⩽ and ⩾ in the AZERTY layout.

Also, on a more personal note, I remember math teachers writing ⩽ and ⩾ on the board, but I don't remember them writing ≤ and ≥ when I was a student. This is definitely not a proof for the spec's intent, but this makes me think that this could be common usage in French.

springcomp commented 1 year ago

@jbrianceau

I have access to a printed version of the NF Z 71-300 French norm from April 2019. Here's what I can tell […]

Thank you so much for digging into this for me. I think we will agree to disagree on the output 😉 as there could be differing interpretations.

On another note, you may be interested to report the missing support for U+2003 EM SPACE in the Windows implementation.

jbrianceau commented 1 year ago

On another note, you may be interested to report the missing support for U+2003 EM SPACE in the Windows implementation.

The "espace cadratin" (U+2003) is part of table 12 (Espaces typographiques) page 20, but it doesn't seem to be referenced anywhere in the AZERTY layout described in appendix A (I wish I had an electronic version of the spec and not a paper one 😅).

It is however part of the BÉPO layout described in appendix B:

Touche Symbole gravé Goupe 1 / Niveau 1 Groupe 1 / Niveau 2 Groupe 2 / Niveau 1 Groupe 2 / Niveau 2
... ... ... ... ... ...
A03 U+2003 (Espace cadratin)

Where should the "espace cadratin" be located in the AZERTY layout?

miloush commented 1 year ago

It sounds like §6.2.3 says "your keyboard has to output these" but doesn't mandate on which keys. So you can still have the layouts in the appendices and meet §6.2.3 which is what the Windows layout is doing (the BÉPO one).

springcomp commented 1 year ago

Where should the "espace cadratin" be located in the AZERTY layout?

My understanding was that Ctrl+Space should emit U+2003 EM SPACE. But you definitely have a more authoritative source than I ever hoped to lay my hands on while authoring the version in this repository.

Slion commented 1 year ago

If we are voting on the "equal to" characters to use I'd rather have the slanted version cause they are a more accurate representation of handwriting. However for smaller font size the non-slanted version are easier to read for my aging eyes. So I also understand the argument about using ≤≥.

Historically I'm guessing the slanted variants only came later as they would have been unreadable on older low res displays.

miloush commented 1 year ago

Historically I'm guessing the slanted variants only came later as they would have been unreadable on older low res displays.

The slanted variant is from Unicode 3.2 (2002) while the normal variant is there pretty much from the beginning.

Only 3 fonts on Windows currently support the slanted characters: Cambria, Cambria Math and Segoe UI Symbol, while pretty much any font supports the normal one.

springcomp commented 1 year ago

If we are voting on the "equal to" characters […]

I’m contacting members of the AFNOR design team to have a feedback. The fact that the text has a conflict is not something I’m comfortable with.

That said, I seem to remember that the document starts with informative text and concludes with normative text. I seem to remember that, indeed, the normative text specifies the slanted variants of the glyphs.

tgrouas commented 1 year ago

I’m contacting members of the AFNOR design team to have a feedback. The fact that the text has a conflict is not something I’m comfortable with.

Hi, I was involved in the french keyboard layout standard comitee.

You point out something that definitely is an issue in the standard. I can fully confirm that the consensus in that comitee was to use U+2A7D ⩽ and U+2A7E ⩾ because that is the symbol most generally used in France while writting with hand, especially in school, high school and university. The character was added in the azerty layout and bepo layout annexes, but the older U+2264 ≤ and U+2265 ≥ were kept by mistake on the common, general list of mandatory characters for both layouts.

Consequences if you want to have a 100% compliant driver:

It may be possible to correct this mistake in a future revision of the standard, either by :

Option n°2 definitely looks like a more compatible and safer update.

Other minor improvements could also be included such as the obligation to provide mathematical symbols such as +, - as index or exponent when using the related dead keys in the azerty layout.

Hope this helps,

Best,

Thibault

springcomp commented 1 year ago

Hi, I was involved in the french keyboard layout standard comitee.

Amen 🤷‍♂️.

And thank you so so much for taking the time providing such a complete and insightful answer.

tgrouas commented 1 year ago

And thank you so so much for taking the time providing such a complete and insightful answer.

Well, thanks to you, and to everybody else contributing to this wonderful and awesome driver that on top of being compliant to the standard is also a joy to use !

Slion commented 1 year ago

The slanted variant is from Unicode 3.2 (2002) while the normal variant is there pretty much from the beginning.

Thanks for looking this up for me 😁

Only 3 fonts on Windows currently support the slanted characters: Cambria, Cambria Math and Segoe UI Symbol, while pretty much any font supports the normal one.

That would be an issue as having a character that won't render on the your keyboard is not very helpful. However in practice it seems to render correctly in most places I tried on Windows 10.

Works in:

Does not work in:

miloush commented 1 year ago

However in practice it seems to render correctly in most places I tried on Windows 10.

That is because of font fallback.

My main concern with this standardization decision would be if Unicode perceives these two characters as having different identities, i.e. having different meaning in some fields of mathematics, and the standard uses the slanted one to represent the normal one because its visual similarity of handwriting in the area. Users will suddenly start creating documents with the slanted characters that did not exist before and would not exist otherwise.

springcomp commented 1 year ago

My main concern with this standardization decision would be [users suddenly being able to] start creating documents with the slanted characters that did not exist before and would not exist otherwise.

On one hand I agree this is a concern. On the other, my intent with this project is to be as much compliant as is possible.

Therefore, I’m inclined to fix this implementation and emit the standardized – slanted – versions, therefore agreeing with the interpretation made by the upcoming Windows implementation.

At the same time, as @tgrouas suggested, I think it would be worthwhile to keed support for those characters. I was thinking they could be moved the European Characters layer with the following key combinations:

What do you think?

Slion commented 1 year ago

I would rather have them in the Greek layers cause they are more related to math than to European characters. The monetary layer would also make more sense than the Eu layer IMHO.

springcomp commented 1 year ago

I would rather have them in the Greek layers cause they are more related to math than to European characters. The monetary layer would also make more sense than the Eu layer IMHO.

Good point. I actually was torn having the same thought. @tgrouas would you have any thoughts on this?

tgrouas commented 1 year ago

@tgrouas would you have any thoughts on this?

yes, indeed. I was thinking first to the Eu character set and here is why :

Maybe there can be better solutions but I feel this one is at least consistent with the aim of the additional character sets in the standard layout, while not being optimal from an ergonomy point of view.

springcomp commented 1 year ago

yes, indeed. I was thinking first to the Eu character set and here is why […]

Thanks for the detailed reasoning.

springcomp commented 1 year ago

Dear all ! @jbrianceau @Slion @tgrouas @miloush.

I’m proud to announce Release 1.8.0.0 which brings all the changes discussed in this issue.

Slion commented 1 year ago

Windows 11 is preventing me to use the downloaded zip fie saying virus detected. Not even sure how to work around that.

springcomp commented 1 year ago

Windows 11 is preventing me to use the downloaded zip fie saying virus detected. Not even sure how to work around that.

Good point. I need to understand how to sign the installer unfortunately.

Did you make sure to unblock the archive before installation ? I’m pretty sure there is a dropdown or small font link that allows you to bypass the SmartScreen filter. Maybe clicking on more information and revealing new options ?

If everything fails, just download the 64-bit MSI and run it directly. The setup.exe is just a two-line C# program that selects either x86 or x64 versions.

Slion commented 1 year ago

I did try the MSI, looks like it ran but it did not update my layout. I'm guessing I need to uninstall the older version first.

springcomp commented 1 year ago

I did try the MSI, looks like it ran but it did not update my layout. I'm guessing I need to uninstall the older version first.

Yes, please, consult the FAQ for upgrading.

Slion commented 1 year ago

Uninstall, reboot, install, reboot, worked. Did not need to set it up anew in the Settings. Works great thanks: ˣxₓ⩽⩽⩾≥≤ Still my infinite symbol is not working but I'm assuming some software is stealing the key somehow… I'm just not sure which one it could.

Slion commented 1 year ago

Still my infinite symbol is not working but I'm assuming some software is stealing the key somehow… I'm just not sure which one it could.

Got it, it was that Nvidia overlay.

springcomp commented 1 year ago

Still my infinite symbol is not working […] Got it, it was that Nvidia overlay.

@Slion would you give me some more information to udpate this page of known broken shortcuts?

Slion commented 1 year ago

I would add Nvidia Shadow Play and Dell Display Manager to that wiki page. Can you give me edit rights to the wiki somehow? Maybe by making me a Collaborator on the project.

springcomp commented 1 year ago

I would add Nvidia Shadow Play and Dell Display Manager to that wiki page. Can you give me edit rights to the wiki somehow? Maybe by making me a Collaborator on the project.

Thanks a lot for proposing. Are you familiar with git and pull requests ? In that case, please fork the Wiki repository and submit your changes as a pull request

Slion commented 1 year ago

Yeah I can do git but I'm too lazy to bother with that process for a couple of edits on a wiki page 😁 that's why I was suggesting direct edit. I guess I could just open an issue with the suggested additions so that you can just copy and paste it in the wiki.

springcomp commented 1 year ago

Yeah I can do git but I'm too lazy to bother with that process for a couple of edits on a wiki page 😁 that's why I was suggesting direct edit. I guess I could just open an issue with the suggested additions so that you can just copy and paste it in the wiki.

OK let me give you write access to the Wiki 🙄

dan-jac commented 1 year ago

Regarding espace cadratin

Where should the "espace cadratin" be located in the AZERTY layout?

My understanding was that Ctrl+Space should emit U+2003 EM SPACE.

I don't think Ctrl+Space is a good choice. Nowhere in the entire norme is Ctrl used for any characters. On Windows, Ctrl is used for keyboard accelerators, and a few legacy control codes in each keyboard layout.

Instead, for inspiration as to where we might put espace cadratin, we should look to the norme's own Annex B, in which the BÉPO layout is defined.

Here, espace cadratin can be found on the space bar on a dead key layer for assorted additional characters: image

This layer is basically the equivalent of AZERTY's AltGr+Eu layer for « Caractères européens additionnels ». In AZERTY, this layer has no additional characters on its space bar. So we could be free to add espace cadratin here.

We couldn't use the exact same position (Shift+Space) as in BÉPO, as the "normal" (non-dead-key) Shift+Space produces a non-unique character (just a regular space, as if you'd pressed Space without modifiers). So we'd have to instead assign espace cadratin to the AltGr position.

Thus, espace cadratin could be typed by pressing: AltGr+Eu; AltGr+Space.

Regarding ¤ U+00A4 CURRENCY SIGN

The Windows implementation does not support AltGr+¤, AltGr+Shift+E to output ¤ U+00A4 CURRENCY SIGN. Although the character is already available using AltGr+¤, Space in both implementations.

My understanding is that the placement of ¤ on AltGr+Shift+E on the monetary symbols layer is a mistake in the norme, and that it is instead meant to be on the same key that you'd use to access the layer – AltGr+F. Thus, you can hold AltGr and double tap F to get this symbol. (Or press AltGr+F followed by space.)

The monetary symbols layer appears to have been “ported” from BÉPO to AZERTY. Here's that layer on BÉPO – it's exactly the same, except that the keys are in their respective BÉPO locations: image

And in BÉPO, putting ¤ on AltGr+Shift+E makes perfect sense, as this is the same position that provides the dead key that is the entry point to the monetary symbols layer.

So the mistake was that when porting the monetary symbols layer from BÉPO to AZERTY, they should have moved ¤ to its matching position for the AZERTY layout – AltGr+F.

Implementing what's written in the norme is impossible on Windows anyway. Because AltGr+Shift+E isn't mapped to anything at all on the base layer of the AZERTY layout, it's not possible for this keystroke to do anything in a dead key layer. It would only be possible if we invented some extra unique character to occupy the AltGr+Shift+E position, which isn't something we'd want to do for a layout shipped in the OS.

springcomp commented 1 year ago

@dan-jac thanks a lot for your very valuable feedback.

I will implement EM SPACE as you suggest and fix the CURRENCY SIGN peculiarity of my layout.

tgrouas commented 1 year ago

Thank you for the bug report. EM SPACE will likely be included in the Eu layer as suggested in the next revision of the standard due this year. Regarding currency sign, it will indeed be moved from erroneous AltGr E to AltGr F in the currency layer to make its input easy with a double tap. Best,

springcomp commented 1 year ago

Thank you for the bug report. EM SPACE will likely be included in the Eu layer as suggested in the next revision of the standard due this year.

Thanks. By any chance, do you know what would be the most likely candidate position?

Regarding currency sign, it will indeed be moved from erroneous AltGr E to AltGr F in the currency layer to make its input easy with a double tap. Best,

Fortunately, the layout already support the standard position. I tried to also support the more arcane position as well. That lead me to create a special dead-key for the AltGr+Shift+E as you rightly pointed out.

tgrouas commented 1 year ago

Thanks. By any chance, do you know what would be the most likely candidate position?

AltGr+Eu; AltGr+Space.

With a double check first to be sure that AltGr+Eu; Space is not technically possible as is it much better from an ergonomy point of view.

Best.