LBSERROR: 'ascii' codec can't decode byte

tpokorra commented 9 years ago

this happens when installing packages on debian wheezy,

Setting up ca-certificates-java (20121112+nmu2) ...
Replacing debian:Certum_Root_CA.pem
Replacing debian:D-TRUST_Root_Class_3_CA_2_2009.pem
Replacing debian:VeriSign_Class_3_Public_Primary_Certification_Authority_-_G5.pem
Replacing debian:AffirmTrust_Networking.pem
Replacing debian:ACEDICOM_Root.pem
Replacing debian:UTN_DATACorp_SGC_Root_CA.pem
Replacing debian:thawte_Primary_Root_CA_-_G2.pem
Replacing debian:E-Guven_Kok_Elektronik_Sertifika_Hizmet_Saglayicisi.pem
Replacing debian:Certigna.pem
Replacing debian:GeoTrust_Primary_Certification_Authority_-_G3.pem
Replacing debian:TDC_Internet_Root_CA.pem
Replacing debian:TWCA_Global_Root_CA.pem
Replacing debian:EE_Certification_Centre_Root_CA.pem
Replacing debian:TC_TrustCenter_Class_2_CA_II.pem
Replacing debian:OISTE_WISeKey_Global_Root_GA_CA.pem
Replacing debian:EBG_Elektronik_Sertifika_Hizmet_Sağlayıcısı.pem
Replacing debian:Buypass_Class_2_CA_1.pem
Replacing debian:America_Online_Root_Certification_Authority_1.pem
...

This failure happens on the line EBG_Elektronik because it contains non ascii characters

[00297] LBSERROR: 'ascii' codec can't decode byte 0xc4 in position 48: ordinal not in range(128)

tpokorra commented 9 years ago

/var/log/lbs.log shows the stacktrace:

Traceback (most recent call last):
  File "/usr/share/lightbuildserver/web/../lib/Build.py", line 94, in buildpackage
    if not self.buildHelper.SetupEnvironment(branchname):
  File "/usr/share/lightbuildserver/web/../lib/BuildHelper.py", line 96, in SetupEnvironment
    if not self.run("cd " + os.path.dirname(setupfile) + "; ./setup.sh " + branchname):
  File "/usr/share/lightbuildserver/web/../lib/BuildHelper.py", line 50, in run
    return self.container.executeInContainer(command)
  File "/usr/share/lightbuildserver/web/../lib/LXCContainer.py", line 86, in executeInContainer
    result = self.shell.executeshell('ssh -f -o "StrictHostKeyChecking no" -o Port=' + self.containerPort + ' -i ' + self.SSHContainerPath + "/container_rsa root@" + self.containerIP + " \"export LANG=C; " + command + " 2>&1 && echo \$?\"")
  File "/usr/share/lightbuildserver/web/../lib/Shell.py", line 39, in executeshell
    line=child.stdout.readline()
  File "/usr/lib64/python3.4/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc4 in position 48: ordinal not in range(128)

tpokorra commented 9 years ago

see for solution: http://stackoverflow.com/questions/3927151/how-to-make-popen-understand-utf-8-properly

hmm, that does not work.

it seems because I call all commands on the build container with LANG=C:

LANG=C locale -k charmap
shows:
charmap="ANSI_X3.4-1968"

That would be good if that was UTF-8

LANG=de_DE.UTF_8 locale -k charmap

does not work. still get ANSI_X2.4-1968

tpokorra commented 9 years ago

it is because of https://docs.python.org/3.2/library/subprocess.html subprocess.popen, universal_newlines = True parameter, that locale.getpreferredencoding() is used

tpokorra commented 9 years ago

solution is to set ENV LANG in /etc/uwsgi.ini

SolidCharity / LightBuildServer

LBSERROR: 'ascii' codec can't decode byte #119