Setting locale to "eo" causes NixOS to fail to build, citing "unknown keysym 'left_double_quotation_mark'"

[JonathanReeve]

Describe the bug

NixOS fails to rebuild, with this error:

error: builder for '/nix/store/gx95y8gjkkb8bi4j5zmvqqgav3qhwgn3-keymap.drv' failed with exit code 1;                                                         
       last 10 log lines:
       > unknown keysym 'left_double_quotation_mark'
       > unknown keysym 'leftquote'
       > unknown keysym 'left_double_quotation_mark'
       > unknown keysym 'leftquote'
       > unknown keysym 'left_double_quotation_mark'
       > unknown keysym 'leftquote'
       > unknown keysym 'left_double_quotation_mark'
       > unknown keysym 'leftquote'
       > unknown keysym 'left_double_quotation_mark'
       > lk_add_key called with bad keycode -1
       For full logs, run 'nix log /nix/store/gx95y8gjkkb8bi4j5zmvqqgav3qhwgn3-keymap.drv'.                   
error: 1 dependencies of derivation '/nix/store/wyrvqdjnw5zaic47g134m681fljc187q-stage-1-init.sh.drv' failed to build   
error: 1 dependencies of derivation '/nix/store/fgnbma8sl3vf0yihy7l7kzcsjg6q1gqz-initrd-linux-5.18.6.drv' failed to build
error: 1 dependencies of derivation '/nix/store/pfhqx1q7n28bkc45l4mn2vhy1vwfalds-nixos-system-jon-laptop-22.11.20220627.020c740.drv' failed to build

Here's the full log, as provided by nix log:

unknown keysym 'leftquote'
unknown keysym 'left_double_quotation_mark'
unknown keysym 'leftquote'
unknown keysym 'left_double_quotation_mark'
unknown keysym 'leftquote'
unknown keysym 'left_double_quotation_mark'
unknown keysym 'leftquote'
unknown keysym 'left_double_quotation_mark'
unknown keysym 'leftquote'
unknown keysym 'left_double_quotation_mark'
unknown keysym 'leftquote'
unknown keysym 'left_double_quotation_mark'
unknown keysym 'leftquote'
unknown keysym 'left_double_quotation_mark'
unknown keysym 'leftquote'
unknown keysym 'left_double_quotation_mark'
unknown keysym 'leftquote'
unknown keysym 'left_double_quotation_mark'
unknown keysym 'leftquote'
unknown keysym 'left_double_quotation_mark'
unknown keysym 'leftquote'
unknown keysym 'left_double_quotation_mark'
unknown keysym 'leftquote'
unknown keysym 'left_double_quotation_mark'
unknown keysym 'leftquote'
unknown keysym 'left_double_quotation_mark'
unknown keysym 'leftquote'
unknown keysym 'left_double_quotation_mark'
unknown keysym 'leftquote'
unknown keysym 'left_double_quotation_mark'
unknown keysym 'leftquote'
unknown keysym 'left_double_quotation_mark'
lk_add_key called with bad keycode -1

I'm not doing anything differently. It just started throwing this error. I'm not setting key symbols anywhere.

My config is here and in the files in that directory.

Steps To Reproduce

Steps to reproduce the behavior:

1. Build Nixos from my config.
2. Notice that it all breaks with the error above.

Expected behavior

A clear and concise description of what you expected to happen.

I expect it not to throw the error above, but to build.

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.18.4, NixOS, 22.11 (Raccoon)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.9.1`
 - channels(jon): `"nixos"`
 - channels(root): `"nixos-21.11"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[JonathanReeve]

I did a bit of digging into this, and it appears that if I unset i18n.defaultLocale, making it the default value instead, it builds fine. So it really doesn't like defaultLocale = "eo"; or defaultLocale = "eo.utf-8". I'll change the title of this issue to reflect that

[wegank]

These lines seem questionable:

https://github.com/NixOS/nixpkgs/blob/5a14883a5f5ea6a1c7b9e875b344c422856be449/nixos/modules/config/console.nix#L11-L19 https://github.com/NixOS/nixpkgs/blob/5a14883a5f5ea6a1c7b9e875b344c422856be449/nixos/modules/config/console.nix#L141-L149

[JonathanReeve]

Checking git blame here to figure out whom to contact. @dasJ , @rnhmjoj , @ju1m : do you know what could be causing this? I suspect it's because "eo" is one of the only languages that doesn't have a region, so the Nixpkgs logic for handing it breaks.

This is a rather serious error, because it means that no one can use NixOS in this language.

[wegank]

I thought that since eo doesn't end with UTF-8, kbd_mode ${if isUnicode then "-u" else "-a"} -C /dev/console actually enforces ASCII, which Esperanto apparently isn't.

[gravndal]

@JonathanReeve have you tried building against a recent commit of nixpkgs?

Building the following with nixos-rebuild build-vm --flake path:(pwd)#esperanto seems to work for me:

{
  inputs = {
    nixpkgs = { url = "github:nixos/nixpkgs/nixos-unstable-small"; };
  };

  outputs = { self, nixpkgs, ... }:
    let
      nixosSystem = args: nixpkgs.lib.nixosSystem (args // rec {
        system = args.system or "x86_64-linux";
        modules = assert (builtins.isList args.modules); [
          ({ ... }: {
            i18n.defaultLocale = "eo";
            console.keyMap = "colemak";
            programs.fish.enable = true;
            services.getty.autologinUser = "root";
          })
        ] ++ args.modules;
      });
    in
    {
      nixosConfigurations = {
        esperanto = nixosSystem {
          modules = [];
        };
      };
    };
}

[wegank]

@gravndal For me, the latest nixos-unstable still doesn't work. I had to add something like this to nixpkgs to be able to switch to Esperanto:

[gravndal]

@wegank that's a shame, then that flake.nix is insufficient to reproduce as it also builds against nixos-unstable...

Though as Esperanto's not the only utf8 locale without the .UTF-8 suffix, that isUnicode predicate really does seem overly naive.

[JonathanReeve]

@gravndal, it still doesn't work for me, on unstable, with all the latest updates. I've just updated my dotfiles if you want to see whether there are any irregularities there, but I suspect it's something with the locales. Maybe unstable-small isn't building the locales? I don't know enough about it to be able to tell.

[JonathanReeve]

For what it's worth, it'd be great to throw in @wegank 's fix above, at least as a stopgap measure.

[wegank]

@gravndal I guess we can patch it like this?

  glibcSupportedLocales = pkgs.runCommand "glibc-supported-locales" { } ''
    unpackFile ${pkgs.glibc.src}
    cat glibc-2*/localedata/SUPPORTED | grep ' \\' | sort -u > $out
  '';

  isUnicode = elem "${toUpper config.i18n.defaultLocale}/UTF-8" (
    splitString " \\\n" (toUpper (readFile "${glibcSupportedLocales}"))
  );

[gravndal]

@JonathanReeve nah, unstable and unstable-small weren't that far apart yesterday so that shouldn't really matter. I was just asking as I can't reproduce with my own configuration (and yours depends on your local fork of home-manager which I was feeling to lazy to sort out at the time :))

@wegank fwiw I was referring to the existing code and not your fix when calling it naive, though apologies if that's how you read that.

[wegank]

@gravndal I'm using a French keyboard, having emdash as a non-ASCII keysym. Could you please test adding services.xserver.layout = "fr";?

By the way, the code above has an IFD, which is not allowed in nixpkgs...

[gravndal]

@wegank still doesn't cause any errors

Edit: what's your console.keyMap?

[wegank]

@gravndal Oh, I use console.useXkbConfig = true;.

[gravndal]

@wegank that triggers it

[SuperSandro2000]

@wegank why did you reopen the issue? Did the PR not fix it?

[wegank]

@SuperSandro2000 The PR was reverted here.

[SuperSandro2000]

Oh, I didn't know because that was silently pushed to master.