userconfig.yaml: Severely malformed YAML: 'NoneType' object is not callable

[staehle]

Hello! Trying to get BuildStream 2.0.1 running on an aarch64 platform (Orange Pi 5) is being a bit of a challenge. I'm mostly just trying to follow the official BuildStream installation instructions.

Simply, I get this error message when trying to do anything with BuildStream:

Error loading user configuration: userconfig.yaml: Severely malformed YAML:

'NoneType' object is not callable

For example here, I'm just trying to build the BuildStream project buildbox-integration so I can get the nice statically-linked BuildBox executables instead of my manually compiled ones!

The host OS/environment does not appear to matter. I get the same error regardless of if I run it natively on this Armbian 23.02.2 Bullseye OS, or inside a buildpack-deps:jammy-scm docker environment (example of which below).

This also occurs if I install BuildStream via pip3, or via the git source:

dockeruser@761236a348a7:~/git/public/buildbox-integration$ pip3 freeze
BuildStream==2.0.1
click==8.1.3
dbus-python==1.2.18
grpcio==1.30.2
grpcio-tools==1.14.1
Jinja2==3.1.2
MarkupSafe==2.1.2
mercurial==6.1.1
pluginbase==1.0.1
protobuf==4.22.1
psutil==5.9.4
PyGObject==3.42.1
pyroaring==0.3.8
python-dateutil==2.8.2
ruamel.yaml==0.17.21
ruamel.yaml.clib==0.2.7
six==1.16.0
ujson==5.7.0

Going off of this example, if I turn bst debug on, I get this:

dockeruser@fc36bea4c406:~/git/public/buildbox-integration$ bst --debug source track --deps all buildbox.bst

Error loading user configuration:

Traceback (most recent call last):
  File "src/buildstream/_yaml.pyx", line 296, in buildstream._yaml.load_data
TypeError: 'NoneType' object is not callable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "src/buildstream/_yaml.pyx", line 273, in buildstream._yaml.load
  File "src/buildstream/_yaml.pyx", line 309, in buildstream._yaml.load_data
buildstream._exceptions.LoadError: Severely malformed YAML:

'NoneType' object is not callable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/spider/.local/lib/python3.10/site-packages/buildstream/_frontend/app.py", line 181, in initialized
    self.context.load(config)
  File "/usr/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/home/spider/.local/lib/python3.10/site-packages/buildstream/_context.py", line 282, in load
    defaults = _yaml.load(_site.default_user_config, shortname="userconfig.yaml")
  File "src/buildstream/_yaml.pyx", line 255, in buildstream._yaml.load
  File "src/buildstream/_yaml.pyx", line 286, in buildstream._yaml.load
buildstream._exceptions.LoadError: userconfig.yaml: Severely malformed YAML:

'NoneType' object is not callable

This example is running inside a buildpack-deps:jammy-scm docker container with some addons, and after manually compiling the required BuildBox executables and installing those to ~/.local/bin:

FROM buildpack-deps:jammy-scm

RUN apt-get update && apt-get install -yq \
    bubblewrap \
    build-essential \
    cmake \
    curl \
    findutils \
    fuse3 \
    git \
    git-lfs \
    googletest \
    grpc++ \
    libfuse3-dev \
    libgmock-dev \
    libgoogle-glog-dev \
    libgrpc-dev \
    libgrpc++-dev \
    libprotobuf-dev \
    libssl-dev \
    lzip \
    ostree \
    patch \
    pigz \
    protobuf-compiler-grpc \
    python3-dev \
    python3-pip \
    python3-setuptools \
    python3-venv \
    quilt \
    sudo \
    tree \
    uuid-dev \
    xzip \
    pkg-config \
    && rm -rf /var/lib/apt/lists/*

RUN mkdir -p /home/spider/

ENV HOME=/home/spider

ENV PATH=/home/spider/.local/bin:$PATH

ENV ARCHI=armv8-a

And running it via:

docker run --rm -i -t \
  -w/home/spider \
  -v$HOME/spiderhome:/home/spider \
  -v$HOME/git:/home/spider/git \
  -u$(id -u):$(id -g)\
  --privileged \
  bbpack:latest

[staehle]

Oddly, as a troubleshooting step, attempting to run BuildStream in the bst2:*-arm64 docker image from FDSDK does successfully startup... but I get a different error message on build -- created a separate issue there for that.

So, how is that project's 'bst' executable able to get past this YAML issue with the same source? The BuildBox infrastructure is a bit confusing

[abderrahim]

Just a quick idea: Do you have a ~/.config/buildstream.conf or ~/.config/buildstream2.conf? It might be the one with malformed syntax?

[staehle]

Hi @abderrahim! Thanks, I did think of that, as I do have a ~/.config/buildstream.conf file tracked in my dotfiles git. However, it works OK as-is on my x86 machine, and removing it entirely didn't affect this output on my aarch64 machine (and the Docker environment I posted uses a new $HOME entirely, so the file wouldn't exist there).

From the Python traceback, it does appear that the error ultimately occurs in _yaml.pyx:296, which is being called from _context.py:282 when trying to load the default userconfig.yaml file. There's nothing wrong with the YAML file, and I'm wondering if it might instead be a problem with the aarch64 version of the ruamel yaml parser that _yaml.pyx is using? That would explain why I cannot run it locally or in my own docker environment, but does not explain why it works correctly in the Buildbox docker for aarch64.

[abderrahim]

As you can see from the line you quoted parser = yaml.CParser(data) it seems to be something related to ruamel.yaml.clib. I don't know if you used wheels or source, maybe the provided wheels are not working correctly?

Please also note that we don't have wheels for aarch64, so you'll need to build buildbox yourself.

[AnotherDaniel]

For what it's worth, I've got the same problem, running the bst tutorial example in a multipass Ubuntu VM on a Apple Silicon Mac.

[abderrahim]

As I mentioned above, this seems to be a problem with ruamel.yaml.clib. To check it, try the following

from ruamel import yaml
yaml.CParser

It should probably raise the correct error that we're after (Here on x86_64, it just shows <class '_ruamel_yaml.CParser'>)