Bootstrap stage #2 build error

[dennishendriksen]

I'm running into the following error when bootstrapping Easybuild:

run_cmd init cmd gcc -v failed:[Errno 2] No such file or directory'

Full logs:

foo:~$ python bootstrap_eb.py $EASYBUILD_PREFIX
[[INFO]] EasyBuild bootstrap script (version 20190922.01, MD5: 55b1295f7b57f138a672a119dcd5514f)
[[INFO]] Found Python 2.7.16 (default, Jan 14 2020, 07:22:06) ; [GCC 8.3.0]

[[INFO]] Installation prefix /home/foo/.local/easybuild
[[INFO]] Found module command '/opt/apps/lmod/lmod/libexec/lmod' via $LMOD_CMD (Lmod), so using it.
[[INFO]] Suitable setuptools installation already found, skipping stage 0...

[[INFO]] +++ STAGE 1: installing EasyBuild in temporary dir with easy_install...

[[INFO]] running pre-install command 'easy_install --quiet --upgrade --prefix=/tmp/tmp4L7k0f/eb_stage1 vsc-install<0.11.4'
[[INFO]] running pre-install command 'easy_install --quiet --upgrade --prefix=/tmp/tmp4L7k0f/eb_stage1 vsc-base<2.9.0'
[[INFO]] installing EasyBuild with 'easy_install --quiet --upgrade --prefix=/tmp/tmp4L7k0f/eb_stage1 easybuild'

[[INFO]] Note: a 'SyntaxError' may be reported for the easybuild/tools/py2vs3/py3.py module.
You can safely ignore this message, it will not affect the functionality of the EasyBuild installation.

  File "build/bdist.linux-x86_64/egg/easybuild/tools/py2vs3/py3.py", line 85
    class WrapperBase(object, metaclass=metaclass):
                                       ^
SyntaxError: invalid syntax

[[INFO]] running post install command 'easy_install --upgrade --prefix=/tmp/tmp4L7k0f/eb_stage1 vsc-base<2.9.0'

[[INFO]] +++ STAGE 2: installing EasyBuild in /home/foo/.local/easybuild with EasyBuild from stage 1...

Traceback (most recent call last):
  File "bootstrap_eb.py", line 1124, in <module>
    main()
  File "bootstrap_eb.py", line 927, in main
    stage2(tmpdir, templates, install_path, distribute_egg_dir, sourcepath)
  File "bootstrap_eb.py", line 790, in stage2
    easybuild_main()
  File "/tmp/tmp4L7k0f/eb_stage1/lib/python2.7/site-packages/easybuild_framework-4.1.1-py2.7.egg/easybuild/main.py", line 197, in main
    init_session_state = session_state()
  File "/tmp/tmp4L7k0f/eb_stage1/lib/python2.7/site-packages/easybuild_framework-4.1.1-py2.7.egg/easybuild/tools/testing.py", line 137, in session_state
    'system_info': get_system_info(),
  File "/tmp/tmp4L7k0f/eb_stage1/lib/python2.7/site-packages/easybuild_framework-4.1.1-py2.7.egg/easybuild/tools/systemtools.py", line 751, in get_system_info
    'gcc_version': get_tool_version('gcc', version_option='-v'),
  File "/tmp/tmp4L7k0f/eb_stage1/lib/python2.7/site-packages/easybuild_framework-4.1.1-py2.7.egg/easybuild/tools/systemtools.py", line 681, in get_tool_version
    trace=False, stream_output=False)
  File "/tmp/tmp4L7k0f/eb_stage1/lib/python2.7/site-packages/easybuild_framework-4.1.1-py2.7.egg/easybuild/tools/run.py", line 88, in cache_aware_func
    res = func(cmd, *args, **kwargs)
  File "/tmp/tmp4L7k0f/eb_stage1/lib/python2.7/site-packages/easybuild_framework-4.1.1-py2.7.egg/easybuild/tools/run.py", line 224, in run_cmd
    raise EasyBuildError("run_cmd init cmd %s failed:%s", cmd, err)
easybuild.tools.build_log.EasyBuildError: 'run_cmd init cmd gcc -v failed:[Errno 2] No such file or directory'

These are the steps I took from an clean Alpine Linux system (failed to get things working on Ubuntu due to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=891541):

su - root
apk update

apk add lua5.3
apk add lua5.3-dev
apk add lua5.3-filesystem
apk add lua5.3-posix
ln -s /usr/bin/lua5.3 /usr/bin/lua
ln -s /usr/bin/luac5.3 /usr/bin/luac
apk add tcl
apk add tcl-dev
apk add wget
wget -O Lmod-8.2.tar.bz2 https://sourceforge.net/projects/lmod/files/Lmod-8.2.tar.bz2/download
tar -xjf Lmod-8.2.tar.bz2
apk add build-base
cd Lmod-8.2.tar.bz2
./configure --prefix=/opt/apps
make install
cp /opt/apps/lmod/lmod/init/profile /etc/profile.d/lmod.sh

apk add python2
# Because of https://github.com/easybuilders/easybuild-framework/issues/3056
apk add py-setuptools
exit

EASYBUILD_PREFIX=$HOME/.local/easybuild
wget https://raw.githubusercontent.com/easybuilders/easybuild-framework/develop/easybuild/scripts/bootstrap_eb.py
python bootstrap_eb.py

Any ideas on the cause of build error or how to solve it?

[ocaisa]

Looks like you don't have a compiler on the system.

[dennishendriksen]

I think I do:

foo:~$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-alpine-linux-musl/8.3.0/lto-wrapper
Target: x86_64-alpine-linux-musl
Configured with: /home/buildozer/aports/main/gcc/src/gcc-8.3.0/configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --build=x86_64-alpine-linux-musl --host=x86_64-alpine-linux-musl --target=x86_64-alpine-linux-musl --with-pkgversion='Alpine 8.3.0' --enable-checking=release --disable-fixed-point --disable-libstdcxx-pch --disable-multilib --disable-nls --disable-werror --disable-symvers --enable-__cxa_atexit --enable-default-pie --enable-default-ssp --enable-cloog-backend --enable-languages=c,c++,objc,fortran,ada --disable-libssp --disable-libmpx --disable-libmudflap --disable-libsanitizer --enable-shared --enable-threads --enable-tls --with-system-zlib --with-linker-hash-style=gnu
Thread model: posix
gcc version 8.3.0 (Alpine 8.3.0)

It was used successfully in a previous 'make install' step.

[ocaisa]

Hmm, the error looks like it is failing to find a command...but not gcc. There are a couple of shell tools required, maybe one is not present? You can turn on debugging for the bootstrap script by exporting EASYBUILD_BOOTSTRAP_DEBUG, maybe that will give more information?

[ocaisa]

Hmm, Alpine had a similar sounding problem in the past, see https://github.com/easybuilders/easybuild-framework/pull/2398

[ocaisa]

@alainvanhoof Maybe you have some tips here?

[dennishendriksen]

Thanks for your quick reply.

Hmm, the error looks like it is failing to find a command...but not gcc. There are a couple of shell tools required, maybe one is not present?

I verified that the required tools are present.

You can turn on debugging for the bootstrap script by exporting EASYBUILD_BOOTSTRAP_DEBUG, maybe that will give more information?

I do see additional logging after export EASYBUILD_BOOTSTRAP_DEBUG=1, but it doesn't appear to be relevant to this issue.

[dennishendriksen]

When I run the following from a Python shell I get the same error:

>>> import subprocess
>>> subprocess.Popen('gcc -v')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/subprocess.py", line 394, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Running the same command with shell=True results in:

>>> subprocess.Popen('gcc -v', shell=True)
<subprocess.Popen object at 0x7f7bfc7d3e10>
>>> Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-alpine-linux-musl/8.3.0/lto-wrapper
Target: x86_64-alpine-linux-musl
Configured with: /home/buildozer/aports/main/gcc/src/gcc-8.3.0/configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --build=x86_64-alpine-linux-musl --host=x86_64-alpine-linux-musl --target=x86_64-alpine-linux-musl --with-pkgversion='Alpine 8.3.0' --enable-checking=release --disable-fixed-point --disable-libstdcxx-pch --disable-multilib --disable-nls --disable-werror --disable-symvers --enable-__cxa_atexit --enable-default-pie --enable-default-ssp --enable-cloog-backend --enable-languages=c,c++,objc,fortran,ada --disable-libssp --disable-libmpx --disable-libmudflap --disable-libsanitizer --enable-shared --enable-threads --enable-tls --with-system-zlib --with-linker-hash-style=gnu
Thread model: posix

Looking at https://github.com/easybuilders/easybuild-framework/blob/develop/easybuild/tools/run.py#L127 it appears that the script is run with shell=True which makes me think that it should work (but it doesn't ...).

Any ideas?

[dennishendriksen]

Fails on Alpine Linux:

subprocess.Popen('gcc -v', shell=True, executable='/bin/bash')

Succeeds after installing bash:

apk add bash

Problem solved!

Apparently bash is a requirement. Could someone add it to https://easybuild.readthedocs.io/en/latest/Installation.html#required-shell-tools?

[Flamefire]

Ah, EasyBuild is using bash explicitely: https://github.com/easybuilders/easybuild-framework/blob/4d4b24ec9ff5216332b063cb8065724a47af364e/easybuild/tools/run.py#L208

Maybe it shouldn't and use the one from "$SHELL" (potentially after matching against valid shells)? What is the default shell on popular systems apart from bash which should work? E.g. what is the one for alpine?

[dennishendriksen]

/bin/ash (https://en.wikipedia.org/wiki/Almquist_shell)

[Flamefire]

Seems this should work. It is however possible that somewhere (even in EasyBlocks) bashism is used. If this is changed to accept other shells, then this should be changed to use sh by default and bash as fallback to detect such issues.

Note that standard Ubuntu sh is dash (variant of ash)