search

microsoft / azurelinux

Linux OS for Azure 1P services and edge appliances
MIT License
4.16k stars 522 forks source link

locale-gen.sh does not work because of missing referenced locales #3041

Open JelteF opened 2 years ago

JelteF commented 2 years ago

The file /usr/share/i18n/locales contains only en_US, but that file references other locale files that are not part of the Mariner installation.

So when you run locale-gen.sh you get the following error:

en_US.ISO-8859-1...[error] cannot open locale definition file `en_GB': No such file or directory

To make locale generation work I had to copy the following locales from my Ubuntu install:

  1. en_GB
  2. i18n
  3. i18n_ctype
  4. iso14651_t1
  5. iso14651_t1_common
  6. translit_* (it was missing many of these, but maybe not all are necessary)
bossmc commented 2 years ago

I just ran into this and found a workaround - install glibc-i18n which (at a whopping 15Mb!) includes all the additional locale definition files.

Still seems like locale-gen.sh should work out of the box without needing to install other packages manually.

Malateshk007 commented 4 months ago

@JelteF, Install the package glibc-i18n since it is not pre-installed. We are closing the issue since it is not a bug.

JelteF commented 4 months ago

I really don't understand why you are not considering this a bug. The en_US.ISO-8859-1 locale does not work!

Malateshk007 commented 4 months ago

@JelteF , as we informed you, we have created internal ADO bug to handle the issue and we are re-opening this issue.

JelteF commented 4 months ago

To document my clarification of the problem here too. The locale-gen.sh command that's included in the base image fails when run.

❯ docker run -it mcr.microsoft.com/cbl-mariner/base/core:2.0 /bin/bash
root [ / ]# locale-gen.sh
Generating locales...
  en_US.ISO-8859-1...[error] cannot open locale definition file `en_GB': No such file or directory

I feel like there are two options to resolve this:

  1. Don't include locale-gen.sh in the base image and only install it when glibc-i18n is installed. And the same should then be done for the file /usr/share/i18n/locales/en_US.
  2. Make sure that locale-gen.sh doesn't fail when run by removing the en_US lines from /etc/locale-gen.conf. (and probably also removing /usr/share/i18n/locales/en_US.
  3. Make sure that locale-gen.sh doesn't fail when run by adding all the necessary files for en_US to be compiled to the base image.
omerfsen commented 1 month ago

Issue is reported on 2022 and now Azure Flexible Servers are using CBL Mariner OS (AzureLinux) and we can't create DBs with UTF8 encoding for en_GB

psql:/tmp/dbSetup.sql:55: ERROR:  encoding "UTF8" does not match locale "en_GB.utf8"
DETAIL:  The chosen LC_CTYPE setting requires encoding "LATIN9".

as collencoding is 16 (it should be 6 for UTF8 but 16 for LATIN9) 


SELECT *                                                                                                                        
FROM pg_collation
WHERE collname like 'en_GB.utf8';
  oid  |  collname  | collnamespace | collowner | collprovider | collisdeterministic | collencoding | collcollate | collctype  | collversion 
-------+------------+---------------+-----------+--------------+---------------------+--------------+-------------+------------+-------------
 12518 | en_GB.utf8 |            11 |        10 | c            | t                   |           16 | en_GB.utf8  | en_GB.utf8 | 2.35
omerf-hmcts commented 1 month ago

Issue is reported on 2022 and now Azure Flexible Servers are using CBL Mariner OS (AzureLinux) and we can't create DBs with UTF8 encoding for en_GB

psql:/tmp/dbSetup.sql:55: ERROR:  encoding "UTF8" does not match locale "en_GB.utf8"
DETAIL:  The chosen LC_CTYPE setting requires encoding "LATIN9".

as collencoding is 16 (it should be 6 for UTF8 but 16 for LATIN9)

SELECT *                                                                                                                        
FROM pg_collation
WHERE collname like 'en_GB.utf8';
  oid  |  collname  | collnamespace | collowner | collprovider | collisdeterministic | collencoding | collcollate | collctype  | collversion 
-------+------------+---------------+-----------+--------------+---------------------+--------------+-------------+------------+-------------
 12518 | en_GB.utf8 |            11 |        10 | c            | t                   |           16 | en_GB.utf8  | en_GB.utf8 | 2.35

As on Azure Flexible Server you can't install a new package (ie. glibc-i18n or others) I think that needs to be handled on Azure side