Find a way to fix R `locale` in Nix for `darwin` and `linux`

[philipp-baumann]

This is to have a universal clean solution, since we did a quick-fix for #48

[b-rodrigues]

so linux and windows should be done with 9390e28e6495f8167f782a17c529949d50c0948c

but now we need to find how to deal with Darwin

[philipp-baumann]

so linux and windows should be done with 9390e28

but now we need to find how to deal with Darwin

great news about the first one! :-)

[philipp-baumann]

@b-rodrigues I'll bump a patch version today

[philipp-baumann]

@b-rodrigues the patch is not working for me, both on Rocky linux and Ubuntu it doesn't.

r$> file.copy(
      # default.nix is the file containing the Nix expression
      from = system.file("extdata", "default.nix", package = "rix"),
      to = ".", overwrite = TRUE
    )

    # nix_build() is a wrapper around the command line tool `nix-build`
    nix_build(project_path = ".")
Launching `nix-build ./default.nix` in blocking mode
Error in sys::exec_internal(cmd = cmd) : 
  Executing 'nix-build' failed with status 127Executing './default.nix' failed with status 127

r$> q()

philipp@philipp-private:~/git/philipp-baumann/rix_tests$ nix-build
error: undefined variable 'glibcLocales'

       at /home/philipp/git/philipp-baumann/rix_tests/default.nix:39:25:

           38|   pkgs.mkShell {
           39|     LOCALE_ARCHIVE = "${glibcLocales}/lib/locale/locale-archive";
             |                         ^
           40|     buildInputs = [ git_archive_pkgs   system_packages  ];

[philipp-baumann]

I am rolling back inst/extdata/default.nix via create_default_nix() back to the last working commit meanwhile. Do you have a solution for the error? Can you reproduce? In case you have the same, we should patch asap or go back to the old working version on master I think.

[philipp-baumann]

@b-rodrigues maybe we can do something like what is done here, but using pkgs.system instead.

https://gitlab.haskell.org/ghc/ghc.nix/-/blob/master/default.nix#L125

From the response in #48 I would assume it works out of the box for macOS and there is no locale issue, but I'm guessing.

in
  pkgs.mkShell {
    LOCALE_ARCHIVE = if pkgs.system == "x86_64-linux"  then "${pkgs.glibcLocales}/lib/locale/locale-archive" else "";
    buildInputs = [ git_archive_pkgs   system_packages  ];
    shellHook = ''R --vanilla'';
  }

[philipp-baumann]

This is some summarizing thread regarding the "nix locale" topic. https://github.com/ES-Nix/locale/issues/1

[philipp-baumann]

https://sourceware.org/glibc/wiki/Proposals/C.UTF-8

[philipp-baumann]

@b-rodrigues actually this is working for my two systems, rocky and ubuntu:

let
 pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/976fa3369d722e76f37c77493d99829540d43845.tar.gz") {};
  git_archive_pkgs = [(pkgs.rPackages.buildRPackage {
    name = "rix";
    src = pkgs.fetchgit {
      url = "https://github.com/b-rodrigues/rix/";
      branchName = "master";
      rev = "ae39d2142461688b1be41db800752a949ebb3c7b";
      sha256 = "sha256-rOhFnc01CzRE6Hl73xGvaE/bQCoX0d5tZ10jzAoqQ7g=";
    };
    propagatedBuildInputs = builtins.attrValues {
      inherit (pkgs.rPackages) httr jsonlite sys;
    };
  }) ];
 tex = (pkgs.texlive.combine {
  inherit (pkgs.texlive) scheme-small ;
});
 system_packages = builtins.attrValues {
  inherit (pkgs) R glibcLocales ;
};
  in
  pkgs.mkShell {
    LOCALE_ARCHIVE = "${pkgs.glibcLocalesUtf8}/lib/locale/locale-archive";
    buildInputs = [ git_archive_pkgs   system_packages  ];
      shellHook = ''R --vanilla'';
  }

What about specifically going for pkgs.glibcLocalesUtf8 config instead of the large pkgs.glibcLocales locale collection? so going in the direction @nviets pointed in the matrix thread.

[philipp-baumann]

@b-rodrigues actually this is working for my two systems, rocky and ubuntu:

let
 pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/976fa3369d722e76f37c77493d99829540d43845.tar.gz") {};
  git_archive_pkgs = [(pkgs.rPackages.buildRPackage {
    name = "rix";
    src = pkgs.fetchgit {
      url = "https://github.com/b-rodrigues/rix/";
      branchName = "master";
      rev = "ae39d2142461688b1be41db800752a949ebb3c7b";
      sha256 = "sha256-rOhFnc01CzRE6Hl73xGvaE/bQCoX0d5tZ10jzAoqQ7g=";
    };
    propagatedBuildInputs = builtins.attrValues {
      inherit (pkgs.rPackages) httr jsonlite sys;
    };
  }) ];
 tex = (pkgs.texlive.combine {
  inherit (pkgs.texlive) scheme-small ;
});
 system_packages = builtins.attrValues {
  inherit (pkgs) R glibcLocales ;
};
  in
  pkgs.mkShell {
    LOCALE_ARCHIVE = "${pkgs.glibcLocalesUtf8}/lib/locale/locale-archive";
    buildInputs = [ git_archive_pkgs   system_packages  ];
      shellHook = ''R --vanilla'';
  }

What about specifically going for pkgs.glibcLocalesUtf8 config instead of the large pkgs.glibcLocales locale collection? so going in the direction @nviets pointed in the matrix thread.

seems like we really need all locales inherited in system_packages, whenever there are some locale envvars that are not set to "en_US.UTF-8" on the host system. Therefore, I am not sure how we can reduce the size. One direction I am imagining is to set the LC_ALL to C.UTF-8 somewhere in Nix, but have not yet managed to do that successfullly.

system_packages = builtins.attrValues {
  inherit (pkgs) R glibcLocales ;
}

[philipp-baumann]

I finally managed to get it working like this, but I had to set the LC_ALL environment variable on the host OS (e.g. Ubuntu) first (the suggestion here was not exactly working, but I adapted the approach: https://nixos.wiki/wiki/Locales).

export LC_ALL=en_US.UTF-8

Then, this works like a charm (we also need a OS-dependent switch in system_packages:

let
 pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/976fa3369d722e76f37c77493d99829540d43845.tar.gz") {};
  git_archive_pkgs = [(pkgs.rPackages.buildRPackage {
    name = "rix";
    src = pkgs.fetchgit {
      url = "https://github.com/b-rodrigues/rix/";
      branchName = "master";
      rev = "ae39d2142461688b1be41db800752a949ebb3c7b";
      sha256 = "sha256-rOhFnc01CzRE6Hl73xGvaE/bQCoX0d5tZ10jzAoqQ7g=";
    };
    propagatedBuildInputs = builtins.attrValues {
      inherit (pkgs.rPackages) httr jsonlite sys;
    };
  }) ];
 tex = (pkgs.texlive.combine {
  inherit (pkgs.texlive) scheme-small ;
});
 system_packages = builtins.attrValues {
  inherit (pkgs) R glibcLocalesUtf8;
};
  in
  pkgs.mkShell {
    LOCALE_ARCHIVE = if pkgs.system == "x86_64-linux"  then "${pkgs.glibcLocalesUtf8}/lib/locale/locale-archive" else "";
    buildInputs = [ git_archive_pkgs   system_packages  ];
      shellHook = "R --vanilla";
  }

[philipp-baumann]

I finally managed to get it working like this, but I had to set the LC_ALL environment variable on the host OS (e.g. Ubuntu) first (the suggestion here was not exactly working, but I adapted the approach: https://nixos.wiki/wiki/Locales).

export LC_ALL=en_US.UTF-8

Then, this works like a charm (we also need a OS-dependent switch in system_packages:

let
 pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/976fa3369d722e76f37c77493d99829540d43845.tar.gz") {};
  git_archive_pkgs = [(pkgs.rPackages.buildRPackage {
    name = "rix";
    src = pkgs.fetchgit {
      url = "https://github.com/b-rodrigues/rix/";
      branchName = "master";
      rev = "ae39d2142461688b1be41db800752a949ebb3c7b";
      sha256 = "sha256-rOhFnc01CzRE6Hl73xGvaE/bQCoX0d5tZ10jzAoqQ7g=";
    };
    propagatedBuildInputs = builtins.attrValues {
      inherit (pkgs.rPackages) httr jsonlite sys;
    };
  }) ];
 tex = (pkgs.texlive.combine {
  inherit (pkgs.texlive) scheme-small ;
});
 system_packages = builtins.attrValues {
  inherit (pkgs) R glibcLocalesUtf8;
};
  in
  pkgs.mkShell {
    LOCALE_ARCHIVE = if pkgs.system == "x86_64-linux"  then "${pkgs.glibcLocalesUtf8}/lib/locale/locale-archive" else "";
    buildInputs = [ git_archive_pkgs   system_packages  ];
      shellHook = "R --vanilla";
  }

I tested on my rocky again today, and setting LC_ALL on Rocky like this did not work.

[philipp-baumann]

However, what is at the moment my favorite solution is explicitly set all necessary locale envvars in pkgs.mkShell as strings.

let
 pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/976fa3369d722e76f37c77493d99829540d43845.tar.gz") {};
  git_archive_pkgs = [(pkgs.rPackages.buildRPackage {
    name = "rix";
    src = pkgs.fetchgit {
      url = "https://github.com/b-rodrigues/rix/";
      branchName = "master";
      rev = "ae39d2142461688b1be41db800752a949ebb3c7b";
      sha256 = "sha256-rOhFnc01CzRE6Hl73xGvaE/bQCoX0d5tZ10jzAoqQ7g=";
    };
    propagatedBuildInputs = builtins.attrValues {
      inherit (pkgs.rPackages) httr jsonlite sys;
    };
  }) ];
 tex = (pkgs.texlive.combine {
  inherit (pkgs.texlive) scheme-small ;
});
 system_packages = builtins.attrValues {
  inherit (pkgs) R glibcLocalesUtf8 ;
};
  in
  pkgs.mkShell {
    LOCALE_ARCHIVE = if pkgs.system == "x86_64-linux" then "${pkgs.glibcLocalesUtf8}/lib/locale/locale-archive" else "";
    LANG = "en_US.UTF-8";
    LC_ALL = "en_US.UTF-8";
    LC_TIME = "en_US.UTF-8";
    LC_MONETARY = "en_US.UTF-8";
    LC_PAPER = "en_US.UTF-8";
    LC_MEASUREMENT = "en_US.UTF-8";
    buildInputs = [ git_archive_pkgs   system_packages  ];
    shellHook = ''R --vanilla'';
  }

[b-rodrigues]

Interesting, we would need to see if this works in CI and also on macOS. Could you set up a macOS runner and so we could see what comes out of it? Also this would simplify the R code quite a bit since we wouldn’t need to generate anything with R

[philipp-baumann]

Interesting, we would need to see if this works in CI and also on macOS. Could you set up a macOS runner and so we could see what comes out of it? Also this would simplify the R code quite a bit since we wouldn’t need to generate anything with R

Unfortunately, it is planned latest end of Q4 this year by GitHub.

[b-rodrigues]

Isn’t that just for Apple Sillicon architecture? macOS on x86_64 is supported https://github.com/actions/runner-images/blob/main/images/macos/macos-13-Readme.md

[philipp-baumann]

Isn’t that just for Apple Sillicon architecture? macOS on x86_64 is supported https://github.com/actions/runner-images/blob/main/images/macos/macos-13-Readme.md

yes, true, let me do that :-)

[philipp-baumann]

Do you know how to make an expression conditional?

Doing this only for macOS, can I just use if then

inherit (pkgs) R glibcLocalesUtf8 ;