perl: warning: Setting locale failed

adrelanos commented 5 years ago

Qubes OS version:

4.0

Affected component(s) or functionality:

Whonix

Steps to reproduce the behavior:

in terminal:

vrms

Expected or desired behavior:

normal output

Actual behavior:

vrms 
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = "",
        LC_ALL = (unset),
        LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").

No non-free or contrib packages installed on host!  rms would be proud.

General notes:

Doesn't only happen (with obscure example) vrms also with many other applications, users reported it happening with youtube-dl.

This randomly started happening. Some update must have introduced it. No Whonix code related to this was changed in years.

A "lazy" fix would be installing package locales-all. But disk space could be pushing it?

After this operation, 130 MB of additional disk space will be used.

Anyone having this issue outside of Whonix?

During template build we have: https://github.com/QubesOS/qubes-builder-debian/blob/master/template_debian/distribution.sh#L354-L358

function updateLocale() {
    debug "Updating locales"
    chroot_cmd localedef -f UTF-8 -i en_US -c en_US.UTF-8
    chroot_cmd update-locale LANG=en_US.UTF-8
}

I wonder why this randomly stops working now?

Nurmagoz commented 5 years ago

The issue exactly happened after installing "qubes-core-agent-passwordless-root " in debian-qubes-minimal

cc @marmarek

unman commented 5 years ago

On Wed, Jun 19, 2019 at 07:35:30AM -0700, TNT BOM BOM wrote:

The issue exactly happened after installing "qubes-core-agent-passwordless-root " in debian-qubes-minimal

No, it's present in the minimal template - nothing to do with installing that package.

Nurmagoz commented 5 years ago

No, it's present in the minimal template - nothing to do with installing that package.

It just showed up after installing it. If you see its not the case , what do you think causing that?

XStreamke commented 5 years ago

Getting the same error when installing whonix-15. Locale info is in /etc/locale.conf, but on installing the new whonix he can't set the locale nl_BE.UTF-8. It's a warning and doesn't block the installation.

adrelanos commented 4 years ago

I haven't seen this issue anymore for a while. No idea why it went away.

This is also the case in these new Qubes-Whonix templates which are currently being tested (but announcement not written yet) (and which might become the new stable version shouldn't there be any other release blockers):

Therefore closing.

Anyone feel free to re-open this if this is still happening.

adrelanos commented 3 years ago

Same issue as before.

locale

locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE=C.UTF-8
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

A messed up locale can be confusing and break scripts (for example breaks str_replace but can be worked around using LANG=C str_replace).

It is now re-happening after Qubes-Whonix 15 (Debian bullseye based) to Qubes-Whonix 16 (bullseye based) release upgrade, Release Upgrade Whonix 15 to Whonix 16.

The issue does not happen in Non-Qubes-Whonix. I therefore suspect there must be something wrong with how Qubes build process creates the Debian base image. https://github.com/QubesOS/qubes-issues/issues/1055 would probably solve that.

sudo locale-gen does not fix the issue.

cat /etc/default/locale

#  File generated by update-locale
LANG=en_US.UTF-8

Output of cat /etc/locale.gen | grep --invert-match \# is empty.

DemiMarie commented 3 years ago

Installing locales-all fixes the problem

adrelanos commented 3 years ago

@DemiMarie

Installing locales-all fixes the problem

I was about to write with concerns about insufficient space on Qubes installer DVD but then saw @marmarek did that already here: https://github.com/QubesOS/qubes-core-agent-linux/pull/322#issuecomment-903579127

marmarek commented 3 years ago

@adrelanos does it happen only on release upgrade from Whonix 15, or on a fresh build of Whonix 16 too?

This is how the locale is generated: https://github.com/QubesOS/qubes-builder-debian/blob/master/template_debian/distribution.sh#L359-L363 I guess it should be also added to /etc/locale.gen, right?

An alternative idea is to set LC_CTYPE=C.UTF-8 instead of LANG=en_US.UTF-8 - to just enable UTF-8 in applications output, but don't specify explicit language. By reading comments in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=609306, it seems setting LANG=C.UTF-8 would be a bad choice.

adrelanos commented 3 years ago

(Added a workaround to Whonix. Referenced by github above. But that's just a band aid. A proper fix would be great.)

@adrelanos does it happen only on release upgrade from Whonix 15, or on a fresh build of Whonix 16 too?

Only on release upgrade from Whonix 15. Fresh build not yet tested. (Will do in near future.)

This is how the locale is generated: https://github.com/QubesOS/qubes-builder-debian/blob/master/template_debian/distribution.sh#L359-L363 I guess it should be also added to /etc/locale.gen, right?

(From https://github.com/grml/grml-debootstrap/issues/25) grml-debootstrap had the same issue (new builds) (https://github.com/grml/grml-debootstrap/issues/25) and the solution was this (https://github.com/grml/grml-debootstrap/commit/430c060be15f60f508428edbc03ea70d9bf04399). Perhaps have a look.

This is how the locale is generated: https://github.com/QubesOS/qubes-builder-debian/blob/master/template_debian/distribution.sh#L359-L363 I guess it should be also added to /etc/locale.gen, right?

Could be. But doing that after the issue happened and then running sudo locale-genand/or sudo update-locale did not fix the issue for me. Maybe it would for new builds but I wouldn't know why.

marmarek commented 3 years ago

Hm, sudo locale-gen should fix that, if en_US.UTF-8 UTF-8 line is uncommented in /etc/locale.gen. Maybe wrong line?

marmarek commented 3 years ago

(From grml/grml-debootstrap#25) grml-debootstrap had the same issue (new builds) (grml/grml-debootstrap#25) and the solution was this (grml/grml-debootstrap@430c060). Perhaps have a look.

So, they have en_US.UTF-8 locale generated by default via /etc/locale.gen. Sounds like we should do that too.

andrewdavidwong commented 2 years ago

Same problem is still happening with a fresh debian-11-minimal template from the repo on a fresh Qubes 4.1.1 installation.

andrewdavidwong commented 2 years ago

Reverted title change based on https://github.com/QubesOS/qubes-core-agent-linux/pull/322#issuecomment-903579127.

DemiMarie commented 2 years ago

We need to figure out what locales the user needs, and generate them via /etc/locale.gen.

Nurmagoz commented 2 years ago

We need to figure out what locales the user needs, and generate them via /etc/locale.gen.

Just use the default which is en_US.UTF-8 no need further expansion.

rocodes commented 4 months ago

I think this could use an affects-4.2 tag as well - on 4.2.2-rc1 I see the same warnings in the journal of my debian 12 minimal qubes and via locale, with a report just like the one @adrelanos mentioned above

ranvik14 commented 3 months ago

I'm still getting this warning with the latest 'debian-12-minimal' template.

@apparatus suggested in Qubes Community Forum [1] that I'll report this here as well.

Here are the logs from my re-test:

Delete 'debian-12-minimal' template - Done.
Install 'debian-12-minimal'template again - OK - See "Log-001".
Clone 'debian-12-minimal' into 'vr-aqt-dl12-00' - Done.
The problem which I faced with minimal templates in Fedora Linux also exist in Debian ...
-> See https://groups.google.com/g/qubes-users/c/m7TfBvVNUsw/m/Vr3rS7g7AwAJ
--> Activate copy & paste for XTerm - OK - See "Log-002".
Install 'qubes-core-agent-passwordless-root' package - OK - See "Log-003".
-> See also https://www.qubes-os.org/doc/templates/minimal/#passwordless-root

### Log-001

    [vr@dom0 ~]$ 
    [vr@dom0 ~]$ sudo qubes-dom0-update qubes-template-debian-12-minimal
    Redirecting to 'qvm-template install  debian-12-minimal'
    Downloading 'qubes-template-debian-12-minimal-0:4.2.0-202308031621'...

    ...

    Installing template 'debian-12-minimal'...
    debian-12-minimal: Importing data
    [vr@dom0 ~]$ 

### Log-002

    user@vr-aqt-dl12-00:~$ 
    user@vr-aqt-dl12-00:~$ cat .Xresources
    XTerm.vt100.selectToClipboard: 1
    user@vr-aqt-dl12-00:~$ 

### Log-003

    root@vr-aqt-dl12-00:~# 
    root@vr-aqt-dl12-00:~# apt update
    Hit:1 https://deb.debian.org/debian bookworm InRelease
    Hit:2 https://deb.debian.org/debian-security bookworm-security InRelease
    Hit:3 https://deb.qubes-os.org/r4.2/vm bookworm InRelease
    Reading package lists... Done                        
    Building dependency tree... Done
    Reading state information... Done
    All packages are up to date.
    root@vr-aqt-dl12-00:~# 
    root@vr-aqt-dl12-00:~# apt install qubes-core-agent-passwordless-root
    Reading package lists... Done
    Building dependency tree... Done
    Reading state information... Done
    The following packages were automatically installed and are no longer required:
      python3-daemon python3-lockfile
    Use 'apt autoremove' to remove them.
    The following NEW packages will be installed:
      qubes-core-agent-passwordless-root
    0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
    Need to get 20.0 kB of archives.
    After this operation, 45.1 kB of additional disk space will be used.
    Get:1 https://deb.qubes-os.org/r4.2/vm bookworm/main amd64 qubes-core-agent-passwordless-root amd64 4.2.36-1+deb12u1 [20.0 kB]
    Fetched 20.0 kB in 0s (52.5 kB/s)                              
    perl: warning: Setting locale failed.
    perl: warning: Please check that your locale settings:
            LANGUAGE = (unset),
            LC_ALL = (unset),
            LANG = "en_US.UTF-8"
        are supported and installed on your system.
    perl: warning: Falling back to the standard locale ("C").
    locale: Cannot set LC_CTYPE to default locale: No such file or directory
    locale: Cannot set LC_MESSAGES to default locale: No such file or directory
    locale: Cannot set LC_ALL to default locale: No such file or directory
    Selecting previously unselected package qubes-core-agent-passwordless-root.
    (Reading database ... 20852 files and directories currently installed.)
    Preparing to unpack .../qubes-core-agent-passwordless-root_4.2.36-1+deb12u1_amd64.deb ...
    Unpacking qubes-core-agent-passwordless-root (4.2.36-1+deb12u1) ...
    Setting up qubes-core-agent-passwordless-root (4.2.36-1+deb12u1) ...
    locale: Cannot set LC_CTYPE to default locale: No such file or directory
    locale: Cannot set LC_MESSAGES to default locale: No such file or directory
    locale: Cannot set LC_ALL to default locale: No such file or directory
    root@vr-aqt-dl12-00:~#

[1] https://forum.qubes-os.org/t/warnings-with-locales-in-debian-12-minimal-based-template/28723

apparatius commented 3 months ago

Downloading 'qubes-template-debian-12-minimal-0:4.2.0-202308031621'...

It seems that the debian-12-minimal template with this change is not available yet.

QubesOS / qubes-issues