search

canonical / cloud-init

Official upstream for the cloud-init: cloud instance initialization
https://cloud-init.io/
Other
2.87k stars 856 forks source link

Locale generation fails on debian #4195

Open TimStallard opened 1 year ago

TimStallard commented 1 year ago

Bug report

We've seen some issues when using cloud-init to set locales on Debian Bookworm. It looks like when a locale is not present, cloud-init attempts to generate it by passing the locale to locale-gen, but locale-gen does not appear to support this. The locale-gen call silently appears to do nothing, but then the later update-locale call fails because the locale has not been generated.

I think instead /etc/locale.gen needs to be updated before running locale-gen - it looks like this is already what some other distro modules use, and a similar fix was made for gentoo on https://github.com/canonical/cloud-init/pull/1205.

Possibly this could also be simplified by using localectl?

Steps to reproduce the problem

Use the following cloud-config to set a locale, on debian bookworm:

#cloud-config
locale: en_GB.UTF-8

Environment details

cloud-init logs

2023-06-23 11:50:53,468 - modules.py[DEBUG]: Running module locale (<module 'cloudinit.config.cc_locale' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_locale.py'>) with frequency once-per-instance
2023-06-23 11:50:53,469 - handlers.py[DEBUG]: start: modules-config/config-locale: running config-locale with frequency once-per-instance
2023-06-23 11:50:53,469 - util.py[DEBUG]: Writing to /var/lib/cloud/instances/vds:tdsbookworm/sem/config_locale - wb: [644] 24 bytes
2023-06-23 11:50:53,469 - helpers.py[DEBUG]: Running config-locale using lock (<FileLock using file '/var/lib/cloud/instances/vds:tdsbookworm/sem/config_locale'>)
2023-06-23 11:50:53,469 - util.py[DEBUG]: Reading from /etc/default/locale (quiet=False)
2023-06-23 11:50:53,469 - util.py[DEBUG]: Read 13 bytes from /etc/default/locale
2023-06-23 11:50:53,469 - cc_locale.py[DEBUG]: Setting locale to en_GB.UTF-8
2023-06-23 11:50:53,469 - debian.py[DEBUG]: Generating locales for en_GB.UTF-8
2023-06-23 11:50:53,469 - subp.py[DEBUG]: Running command ['locale-gen', 'en_GB.UTF-8'] with allowed return codes [0] (shell=False, capture=False)
2023-06-23 11:50:53,488 - debian.py[DEBUG]: Updating /etc/default/locale with locale setting LANG=en_GB.UTF-8
2023-06-23 11:50:53,488 - subp.py[DEBUG]: Running command ['update-locale', '--locale-file=/etc/default/locale', 'LANG=en_GB.UTF-8'] with allowed return codes [0] (shell=False, capture=False)
2023-06-23 11:50:53,533 - handlers.py[DEBUG]: finish: modules-config/config-locale: FAIL: running config-locale with frequency once-per-instance
2023-06-23 11:50:53,534 - util.py[WARNING]: Running module locale (<module 'cloudinit.config.cc_locale' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_locale.py'>) failed
2023-06-23 11:50:53,534 - util.py[DEBUG]: Running module locale (<module 'cloudinit.config.cc_locale' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_locale.py'>) failed
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/cloudinit/config/modules.py", line 246, in _run_modules
    ran, _r = cc.run(
              ^^^^^^^
  File "/usr/lib/python3/dist-packages/cloudinit/cloud.py", line 67, in run
    return self._runners.run(name, functor, args, freq, clear_on_fail)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 185, in run
    results = functor(*args)
              ^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_locale.py", line 70, in handle
    cloud.distro.apply_locale(locale, locale_cfgfile)
  File "/usr/lib/python3/dist-packages/cloudinit/distros/debian.py", line 132, in apply_locale
    update_locale_conf(locale, out_fn, keyname=keyname)
  File "/usr/lib/python3/dist-packages/cloudinit/distros/debian.py", line 374, in update_locale_conf
    subp.subp(
  File "/usr/lib/python3/dist-packages/cloudinit/subp.py", line 335, in subp
    raise ProcessExecutionError(
cloudinit.subp.ProcessExecutionError: Unexpected error while running command.
Command: ['update-locale', '--locale-file=/etc/default/locale', 'LANG=en_GB.UTF-8']
Exit code: 255
Reason: -
Stdout: -
Stderr: -
blackboxsw commented 1 year ago

Confirmed this on LXD:

cat > ud.yaml <<EOF
#cloud-config  
locale: en_GB.UTF-8
EOF
lxc launch images:debian/bookworm -c user.user-data="$(cat ud.yaml)" test-bookwork
lxc exec test-bookworm bash
# apt install cloud-init
# sudo reboot 
# cloud-init status --long
status: error
boot_status_code: enabled-by-generator
last_update: Fri, 23 Jun 2023 13:06:12 +0000
detail:
('locale', ProcessExecutionError("Unexpected error while running command.\nCommand: ['update-locale', '--locale-file=/etc/default/locale', 'LANG=en_GB.UTF-8']\nExit code: 255\nReason: -\nStdout: -\nStderr: -"))
blackboxsw commented 1 year ago

cloud-init.tar.gz

blackboxsw commented 1 year ago

Debian docs for locale

garryshield commented 4 months ago

i get the same err when i set locale: zh_CN.UTF-8

$ sudo cloud-init --version
/usr/bin/cloud-init 22.4.2

$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
$ cat /var/log/cloud-init.log | grep locale
2024-05-15 02:11:21,791 - modules.py[DEBUG]: Running module locale (<module 'cloudinit.config.cc_locale' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_locale.py'>) with frequency once-per-instance
2024-05-15 02:11:21,791 - handlers.py[DEBUG]: start: modules-config/config-locale: running config-locale with frequency once-per-instance
2024-05-15 02:11:21,791 - util.py[DEBUG]: Writing to /var/lib/cloud/instances/2ed7f3503fed5f25314ce8304e12dc23a8906cf6/sem/config_locale - wb: [644] 23 bytes
2024-05-15 02:11:21,791 - helpers.py[DEBUG]: Running config-locale using lock (<FileLock using file '/var/lib/cloud/instances/2ed7f3503fed5f25314ce8304e12dc23a8906cf6/sem/config_locale'>)
2024-05-15 02:11:21,792 - util.py[DEBUG]: Reading from /etc/default/locale (quiet=False)
2024-05-15 02:11:21,792 - util.py[DEBUG]: Read 13 bytes from /etc/default/locale
2024-05-15 02:11:21,792 - cc_locale.py[DEBUG]: Setting locale to zh_CN.UTF-8
2024-05-15 02:11:21,792 - debian.py[DEBUG]: Generating locales for zh_CN.UTF-8
2024-05-15 02:11:21,792 - subp.py[DEBUG]: Running command ['locale-gen', 'zh_CN.UTF-8'] with allowed return codes [0] (shell=False, capture=False)
2024-05-15 02:11:21,809 - debian.py[DEBUG]: Updating /etc/default/locale with locale setting LANG=zh_CN.UTF-8
2024-05-15 02:11:21,809 - subp.py[DEBUG]: Running command ['update-locale', '--locale-file=/etc/default/locale', 'LANG=zh_CN.UTF-8'] with allowed return codes [0] (shell=False, capture=False)
2024-05-15 02:11:21,868 - handlers.py[DEBUG]: finish: modules-config/config-locale: FAIL: running config-locale with frequency once-per-instance
2024-05-15 02:11:21,868 - util.py[WARNING]: Running module locale (<module 'cloudinit.config.cc_locale' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_locale.py'>) failed
2024-05-15 02:11:21,868 - util.py[DEBUG]: Running module locale (<module 'cloudinit.config.cc_locale' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_locale.py'>) failed
  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_locale.py", line 70, in handle
    cloud.distro.apply_locale(locale, locale_cfgfile)
  File "/usr/lib/python3/dist-packages/cloudinit/distros/debian.py", line 132, in apply_locale
    update_locale_conf(locale, out_fn, keyname=keyname)
  File "/usr/lib/python3/dist-packages/cloudinit/distros/debian.py", line 374, in update_locale_conf
Command: ['update-locale', '--locale-file=/etc/default/locale', 'LANG=zh_CN.UTF-8']

in debian locale-gen direct read /etc/locale.gen file for enabled list and ignore the params you can check the source code /usr/sbin/locale-gen

locale-gen zh_CN.UTF-8

i think this function regenerate_locale shoud do something like

sed -ie 's/# zh_CN.UTF-8 UTF-8/zh_CN.UTF-8 UTF-8/g' /etc/locale.gen

then call locale-gen

or use bootcmd for this

pranaysy commented 3 months ago

I get the same error as @garryshield on the exact same debian 12 config, but with locale: en_US.UTF-8