ExpansionError during parsing /home/mender/poky/meta/recipes-extended/images/core-image-testmaster.bb

[laibulle]

steps to reproduce

useradd -m mender
su mender
cd
git clone -b pyro http://git.yoctoproject.org/git/poky
cd poky
git clone -b pyro git://git.yoctoproject.org/meta-raspberrypi
git clone -b pyro git://github.com/mendersoftware/meta-mender
git clone -b pyro git://git.openembedded.org/meta-openembedded
source oe-init-build-env rpi3_build
bitbake-layers add-layer ../meta-mender/meta-mender-core
bitbake-layers add-layer ../meta-mender/meta-mender-demo
bitbake rpi-basic-image

conf/bblayers.conf

# POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
POKY_BBLAYERS_CONF_VERSION = "2"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
  /home/mender/poky/meta \
  /home/mender/poky/meta-poky \
  /home/mender/poky/meta-yocto-bsp \
  /home/mender/poky/meta-raspberrypi \
  /home/mender/poky/meta-openembedded/meta-oe \
  /home/mender/poky/meta-openembedded/meta-multimedia \
  /home/mender/poky/meta-openembedded/meta-networking \
  /home/mender/poky/meta-openembedded/meta-python \
  /home/mender/poky/meta-mender/meta-mender-core \
  /home/mender/poky/meta-mender/meta-mender-demo \
  "

Error

WARNING: /home/mender/poky/meta/recipes-extended/images/core-image-testmaster.bb: Exception during build_dependencies for create_shar#################################                                                         | ETA:  0:02:28
WARNING: /home/mender/poky/meta/recipes-extended/images/core-image-testmaster.bb: Error during finalise of /home/mender/poky/meta/recipes-extended/images/core-image-testmaster.bb
ERROR: ExpansionError during parsing /home/mender/poky/meta/recipes-extended/images/core-image-testmaster.bb
Traceback (most recent call last):
bb.data_smart.ExpansionError: Failure expanding variable create_shar, expression was    # copy in the template shar extractor script
    cp /home/mender/poky/meta/files/toolchain-shar-extract.sh /home/mender/poky/rpi3_build/tmp/work/raspberrypi0-poky-linux-gnueabi/core-image-testmaster/1.0-r0/x86_64-deploy-core-image-testmaster-populate-sdk/poky-glibc-x86_64-core-image-testmaster-arm1176jzfshf-vfp-toolchain-2.3.2.sh

    rm -f /home/mender/poky/rpi3_build/tmp/work/raspberrypi0-poky-linux-gnueabi/core-image-testmaster/1.0-r0/temp/pre_install_command /home/mender/poky/rpi3_build/tmp/work/raspberrypi0-poky-linux-gnueabi/core-image-testmaster/1.0-r0/temp/post_install_command

    if [ 1 -eq 1 ] ; then
        cp /home/mender/poky/meta/files/toolchain-shar-relocate.sh /home/mender/poky/rpi3_build/tmp/work/raspberrypi0-poky-linux-gnueabi/core-image-testmaster/1.0-r0/temp/post_install_command
    fi
    cat << "EOF" >> /home/mender/poky/rpi3_build/tmp/work/raspberrypi0-poky-linux-gnueabi/core-image-testmaster/1.0-r0/temp/pre_install_command

EOF

    cat << "EOF" >> /home/mender/poky/rpi3_build/tmp/work/raspberrypi0-poky-linux-gnueabi/core-image-testmaster/1.0-r0/temp/post_install_command

EOF
    sed -i -e '/@SDK_PRE_INSTALL_COMMAND@/r /home/mender/poky/rpi3_build/tmp/work/raspberrypi0-poky-linux-gnueabi/core-image-testmaster/1.0-r0/temp/pre_install_command' \
        -e '/@SDK_POST_INSTALL_COMMAND@/r /home/mender/poky/rpi3_build/tmp/work/raspberrypi0-poky-linux-gnueabi/core-image-testmaster/1.0-r0/temp/post_install_command' \
        /home/mender/poky/rpi3_build/tmp/work/raspberrypi0-poky-linux-gnueabi/core-image-testmaster/1.0-r0/x86_64-deploy-core-image-testmaster-populate-sdk/poky-glibc-x86_64-core-image-testmaster-arm1176jzfshf-vfp-toolchain-2.3.2.sh

    # substitute variables
    sed -i -e 's#@SDK_ARCH@#x86_64#g' \
        -e 's#@SDKPATH@#/opt/poky/2.3.2#g' \
        -e 's#@SDKEXTPATH@#~/poky_sdk#g' \
        -e 's#@OLDEST_KERNEL@#2.6.32#g' \
        -e 's#@REAL_MULTIMACH_TARGET_SYS@#arm1176jzfshf-vfp-poky-linux-gnueabi#g' \
        -e 's#@SDK_TITLE@#${@d.getVar("SDK_TITLE").replace('&', '\&')}#g' \
        -e 's#@SDK_VERSION@#2.3.2#g' \
        -e '/@SDK_PRE_INSTALL_COMMAND@/d' \
        -e '/@SDK_POST_INSTALL_COMMAND@/d' \
        -e 's#@SDK_GCC_VER@#${@oe.utils.host_gcc_version(d)}#g' \
        /home/mender/poky/rpi3_build/tmp/work/raspberrypi0-poky-linux-gnueabi/core-image-testmaster/1.0-r0/x86_64-deploy-core-image-testmaster-populate-sdk/poky-glibc-x86_64-core-image-testmaster-arm1176jzfshf-vfp-toolchain-2.3.2.sh

    # add execution permission
    chmod +x /home/mender/poky/rpi3_build/tmp/work/raspberrypi0-poky-linux-gnueabi/core-image-testmaster/1.0-r0/x86_64-deploy-core-image-testmaster-populate-sdk/poky-glibc-x86_64-core-image-testmaster-arm1176jzfshf-vfp-toolchain-2.3.2.sh
# There are also the following hardware board target machines included for

    # append the SDK tarball
    cat /home/mender/poky/rpi3_build/tmp/work/raspberrypi0-poky-linux-gnueabi/core-image-testmaster/1.0-r0/x86_64-deploy-core-image-testmaster-populate-sdk/poky-glibc-x86_64-core-image-testmaster-arm1176jzfshf-vfp-toolchain-2.3.2.tar.xz >> /home/mender/poky/rpi3_build/tmp/work/raspberrypi0-poky-linux-gnueabi/core-image-testmaster/1.0-r0/x86_64-deploy-core-image-testmaster-populate-sdk/poky-glibc-x86_64-core-image-testmaster-arm1176jzfshf-vfp-toolchain-2.3.2.sh

    # delete the old tarball, we don't need it anymore
    rm /home/mender/poky/rpi3_build/tmp/work/raspberrypi0-poky-linux-gnueabi/core-image-testmaster/1.0-r0/x86_64-deploy-core-image-testmaster-populate-sdk/poky-glibc-x86_64-core-image-testmaster-arm1176jzfshf-vfp-toolchain-2.3.2.tar.xz
 which triggered exception OSError: [Errno 12] Cannot allocate memory

Summary: There were 2 WARNING messages shown.
Summary: There was 1 ERROR message shown, returning a non-zero exit code.

[pbrkr]

The issue here seems to be the exception "OSError: [Errno 12] Cannot allocate memory". I'd guess that's a problem with your build machine rather than within OE (unless there's a bug in bitbake we don't know about).

Do you have at least ~1GB RAM on the machine you're building on? And do you have an alternative machine you could test this on?

[laibulle]

I will try on an other VPS.

[agherzan]

What are the specs of your vps?

[laibulle]

It has 2 GB of ram and 2 x86_64 cores. But it already run a full Mender stack so I don't think that it has 1 GB of ram available

free -m

              total        used        free      shared  buff/cache   available
Mem:           2001         753         152          36        1095         942
Swap:             0           0           0

[pbrkr]

Could you try building core-image-minimal with a clean poky clone, without any additional layers? Then add layers in a couple at a time (meta-oe layers first, then meta-raspberrypi, then meta-mender layers).

[agherzan]

Even so, I'm pretty sure less than 1G of ram is not feasible.

[laibulle]

I am trying on a baremetal server, it seems to work for now. I will give you a feedback once it is completed.

Thank you

[agherzan]

Sounds good.

[laibulle]

Ok, I was able to build with a better server.

[agherzan]

Thanks for feedback. I'm glad. Enjoy!