Unable to install easybuild using the pip-easybuild

[baloe]

Following your instructions here I am trying to install easybuild as a module.

I have already installed lmod.

I thus run

source /etc/profile.d/modules.sh
export EB_TMPDIR=/tmp/$USER/eb_tmp
python3 -m pip install --ignore-installed --prefix $EB_TMPDIR easybuild
export PATH=$EB_TMPDIR/bin:$PATH
export PYTHONPATH=$(/bin/ls -rtd -1 $EB_TMPDIR/lib*/python*/site-packages | tail -1):$PYTHONPATH
export EB_PYTHON=python3
eb --install-latest-eb-release --prefix $PATH_MODULES

There is no error installing eb with pip, but the last line fails with this error:

ERROR: Build of /tmp/eb-03gzr7y4/tmpux2d2apf/easybuilders/easybuild-easyconfigs-develop/easybuild/easyconfigs/e/EasyBuild/EasyBuild-4.6.0.eb failed (err: 'build failed (first 300 chars): Sanity check failed: command "/usr/bin/python3 -c "import easybuild"" failed; output:\nTraceback (most recent call last):\n  File "<string>", line 1, in <module>\nModuleNotFoundError: No module named \'easybuild\'\nsanity check command eb --version exited with code 1 (output: ERROR: No compatible \'python\'')

Here is the log easybuild-EasyBuild-4.6.0-20220823.161750.GBhfp.log

Why does the log say

== 2022-08-23 16:17:53,307 environment.py:91 INFO Environment variable PYTHONPATH set to /soft/modules/software/EasyBuild/4.6.0/lib/python3.10/site-packages (previously undefined)

? I am exporting the proper PYTHONPATH above.

[posixcompliance]

I am also having similar issues in the steps before this. Using lmod, command module load EasyBuild fails to find a module named "EasyBuild" after the running sudo pip install easybuild to install system-wide.

[Asheswright]

I have also hit this problem. It appears only to affect python3.10. Python3.6 and python3.9 work fine.

[boegel]

The failing eb commands that mention "No compatible 'python' command found via $PATH" are a bit misleading, this is the actual problem:

== 2022-08-23 16:18:19,245 build_log.py:169 ERROR EasyBuild crashed with an error (at easybuild/base/exceptions.py:124 in __init__): Sanity check failed: command "/usr/bin/python3 -c "import easybuild"" failed; output:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'easybuild'

For some reason, the easybuild Python package can not be imported.

I'm looking into this exact same problem now in an environment where a very recent version of setuptools is being used.

@baloe @Ezimkin @Asheswright Did you experience these problems with a recent version of setuptools too? Which version are you running? You should be able to tell using the following command:

python -c 'import setuptools; print(setuptools.version.__version__)'

[boegel]

It looks like the culprit here could be that: i) recent versions of setuptools, probably >= 60.0.0 which use their own copy of distutils, somehow make setup.py install install .egg directories rather than installing directly into the site-packages directory; ii) Since setuptools v49.0.0, .egg directories are no longer automatically added to the Python search path, see also https://github.com/easybuilders/easybuild-easyblocks/issues/2693 and https://github.com/pypa/setuptools/pull/2166;

@baloe @Ezimkin @Asheswright Next to checking which version of setuptools you were using, can you also check what the contents of the EasyBuild installation directory (like /soft/modules/software/EasyBuild/4.6.0/lib/python3.10/site-packages) is?

[boegel]

As long as a sufficiently recent version of pip is installed, the following command should work to install the latest EasyBuild release using pip install rather than setup.py install:

eb --install-latest-eb-release --try-amend=use_pip=1

We will look into enhancing the easyblock that is used to install EasyBuild so it automatically favors installing with pip install rather than setup.py install when it makes sense (sufficiently recent Python, pip, etc.)...

[boegel]

If anyone runs into similar issues, please try using the updated EasyBuildMeta easyblock from https://github.com/easybuilders/easybuild-easyblocks/pull/2805, using:

eb --include-easyblocks-from-pr 2805 ...