search

aivazis / config

10 stars 2 forks source link

mm file encodings #5

Closed rtburns-jpl closed 5 years ago

rtburns-jpl commented 5 years ago

I can't compile pyre under config on Ubuntu 18.04 with Python 3.6 due to encoding errors. The mm makefiles are detected as ascii-encoded but use utf-8 characters.

Example Dockerfile:

FROM ubuntu:18.04

# build-required packages
RUN apt-get update \
 && apt-get install -y git python3 make g++

# set up home directory for build
RUN git clone https://github.com/aivazis/config /config \
 && git clone https://github.com/pyre/pyre      /pyre

RUN mkdir /pyre-install \
 && cd /pyre/lib \
 && BLD_CONFIG=/config \
    EXPORT_ROOT=/pyre-install \
    PYTHON=python3.6m \
    PYTHON_INCDIR=/usr/include/python3.6m \
    PYTHON_LIB=python3.6m \
    PYTHON_LIBDIR=/usr/lib/python3.6m \
    PYTHON_PYCFLAGS=-b \
    /config/make/mm.py

Python traceback:

Traceback (most recent call last):
  File "/config/make/mm.py", line 1025, in <module>
    status = run()
  File "/config/make/mm.py", line 983, in run
    builder = Builder()
  File "/config/make/mm.py", line 926, in __init__
    self.requirements = self.getProjectRequirements()
  File "/config/make/mm.py", line 531, in getProjectRequirements
    symbols = self._loadSymbols(path=reqfile)
  File "/config/make/mm.py", line 955, in _loadSymbols
    exec(stream.read(), symbols)
  File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 42: ordinal
 not in range(128)

I can compile if I add the keyword encoding="utf-8" to the open() call on line 947 of mm.py. Should mm.py autodetect the makefile encoding, or is it fine to assume utf-8?

aivazis commented 5 years ago

Sounds like the environment is missing the LANG variable. It should be set to en_US.utf-8

-- Michael

On Jun 11, 2019, at 9:37 PM, Ryan Burns notifications@github.com wrote:

I can't compile pyre under config on Ubuntu 18.04 with Python 3.6 due to encoding errors. The mm makefiles are detected as ascii-encoded but use utf-8 characters.

Example Dockerfile:

FROM ubuntu:18.04

build-required packages

RUN apt-get update \ && apt-get install -y git python3 make g++

set up home directory for build

RUN git clone https://github.com/aivazis/config /config \ && git clone https://github.com/pyre/pyre /pyre

RUN mkdir /pyre-install \ && cd /pyre/lib \ && BLD_CONFIG=/config \ EXPORT_ROOT=/pyre-install \ PYTHON=python3.6m \ PYTHON_INCDIR=/usr/include/python3.6m \ PYTHON_LIB=python3.6m \ PYTHON_LIBDIR=/usr/lib/python3.6m \ PYTHON_PYCFLAGS=-b \ /config/make/mm.py Python traceback:

Traceback (most recent call last): File "/config/make/mm.py", line 1025, in status = run() File "/config/make/mm.py", line 983, in run builder = Builder() File "/config/make/mm.py", line 926, in init self.requirements = self.getProjectRequirements() File "/config/make/mm.py", line 531, in getProjectRequirements symbols = self._loadSymbols(path=reqfile) File "/config/make/mm.py", line 955, in _loadSymbols exec(stream.read(), symbols) File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode return codecs.ascii_decode(input, self.errors)[0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 42: ordinal not in range(128) I can compile if I add the keyword encoding="utf-8" to the open() call on line 947 of mm.py. Should mm.py autodetect the makefile encoding, or is it fine to assume utf-8?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

rtburns-jpl commented 5 years ago

Ah, thanks. The Ubuntu base images don't have any locales set up so that fixed it.