search

easybuilders / easybuild-framework

EasyBuild is a software installation framework in Python that allows you to install software in a structured and robust way.
https://easybuild.io
GNU General Public License v2.0
152 stars 202 forks source link

postinstallpatches fail #4040

Closed surak closed 2 years ago

surak commented 2 years ago

When using postinstallpatches = ['file.patch'], I get this strange error:

== 2022-07-05 22:43:20,660 filetools.py:1983 INFO Removing lock /p/home/jusers/strube1/juwels/easybuild/juwelsbooster/software/.locks/_p_home_jusers_strube1_juwels_easybuild_juwelsbooster_software_TensorFlow_2.6.0-gcccoremkl-11.2.0-2021.4.0-CUDA-11.5.lock...
== 2022-07-05 22:43:21,042 filetools.py:382 INFO Path /p/home/jusers/strube1/juwels/easybuild/juwelsbooster/software/.locks/_p_home_jusers_strube1_juwels_easybuild_juwelsbooster_software_TensorFlow_2.6.0-gcccoremkl-11.2.0-2021.4.0-CUDA-11.5.lock successfully removed.
== 2022-07-05 22:43:21,042 filetools.py:1987 INFO Lock removed: /p/home/jusers/strube1/juwels/easybuild/juwelsbooster/software/.locks/_p_home_jusers_strube1_juwels_easybuild_juwelsbooster_software_TensorFlow_2.6.0-gcccoremkl-11.2.0-2021.4.0-CUDA-11.5.lock
== 2022-07-05 22:43:21,106 build_log.py:169 ERROR EasyBuild crashed with an error (at easybuild/base/exceptions.py:124 in __init__): Traceback (most recent call last):
  File "/p/software/juwelsbooster/stages/2022/software/EasyBuild/4.5.5/lib/python3.6/site-packages/easybuild/main.py", line 128, in build_and_install_software
    (ec_res['success'], app_log, err) = build_and_install_one(ec, init_env)
  File "/p/software/juwelsbooster/stages/2022/software/EasyBuild/4.5.5/lib/python3.6/site-packages/easybuild/framework/easyblock.py", line 4058, in build_and_install_one
    result = app.run_all_steps(run_test_cases=run_test_cases)
  File "/p/software/juwelsbooster/stages/2022/software/EasyBuild/4.5.5/lib/python3.6/site-packages/easybuild/framework/easyblock.py", line 3941, in run_all_steps
    self.run_step(step_name, step_methods)
  File "/p/software/juwelsbooster/stages/2022/software/EasyBuild/4.5.5/lib/python3.6/site-packages/easybuild/framework/easyblock.py", line 3776, in run_step
    step_method(self)()
  File "/p/software/juwelsbooster/stages/2022/software/EasyBuild/4.5.5/lib/python3.6/site-packages/easybuild/framework/easyblock.py", line 2868, in post_install_step
    self.apply_post_install_patches()
  File "/p/software/juwelsbooster/stages/2022/software/EasyBuild/4.5.5/lib/python3.6/site-packages/easybuild/framework/easyblock.py", line 2859, in apply_post_install_patches
    self.patch_step(beginpath=self.installdir, patches=patches)
TypeError: patch_step() got an unexpected keyword argument 'beginpath'
 (at easybuild/main.py:167 in build_and_install_software)
ERROR: Traceback (most recent call last):
  File "/p/software/juwelsbooster/stages/2022/software/EasyBuild/4.5.5/lib/python3.6/site-packages/easybuild/main.py", line 128, in build_and_install_software
    (ec_res['success'], app_log, err) = build_and_install_one(ec, init_env)
  File "/p/software/juwelsbooster/stages/2022/software/EasyBuild/4.5.5/lib/python3.6/site-packages/easybuild/framework/easyblock.py", line 4058, in build_and_install_one
    result = app.run_all_steps(run_test_cases=run_test_cases)
  File "/p/software/juwelsbooster/stages/2022/software/EasyBuild/4.5.5/lib/python3.6/site-packages/easybuild/framework/easyblock.py", line 3941, in run_all_steps
    self.run_step(step_name, step_methods)
  File "/p/software/juwelsbooster/stages/2022/software/EasyBuild/4.5.5/lib/python3.6/site-packages/easybuild/framework/easyblock.py", line 3776, in run_step
    step_method(self)()
  File "/p/software/juwelsbooster/stages/2022/software/EasyBuild/4.5.5/lib/python3.6/site-packages/easybuild/framework/easyblock.py", line 2868, in post_install_step
    self.apply_post_install_patches()
  File "/p/software/juwelsbooster/stages/2022/software/EasyBuild/4.5.5/lib/python3.6/site-packages/easybuild/framework/easyblock.py", line 2859, in apply_post_install_patches
    self.patch_step(beginpath=self.installdir, patches=patches)
TypeError: patch_step() got an unexpected keyword argument 'beginpath'
boegel commented 2 years ago

@surak Can you share your EasyBuild configuration (output of eb --show-config)? Do you have any custom easyblokcs in place customize patch_step that may explain why you're hitting this?

surak commented 2 years ago

I don't think we have something like that on patch_step.


#
# Current EasyBuild configuration
# (C: command line argument, D: default value, E: environment variable, F: configuration file)
#
allow-loaded-modules          (E) = EasyBuild
buildpath                     (E) = /dev/shm/strube1/juwelsbooster
containerpath                 (E) = /p/project/ccstao/cstao05/easybuild/juwelsbooster/containers
cuda-compute-capabilities     (E) = 8.0
detect-loaded-modules         (E) = error
experimental                  (E) = True
hide-deps                     (E) = ACTC, ANTLR, APR, APR-util, AT-SPI2-ATK, AT-SPI2-core, ATK, Autoconf, Automake, Bison, Blosc, CUSP, Coreutils, CubeWriter, DB, DBus, DocBook-XML, Dyninst, ETSF_IO, Exiv2, FFmpeg, FLTK, FTGL, FoX, GCCcore, GEGL, GL2PS, GLEW, GLM, GLPK, GLib, GObject-Introspection, GPC, GStreamer, GTI, GTK+, GTS, Gdk-Pixbuf, Ghostscript, GraphicsMagick, GtkSourceView, HarfBuzz, ICU, ISL, JSON-C, JSON-GLib, JUnit, JasPer, JsonCpp, JupyterKernel-Bash, JupyterKernel-Cling, JupyterKernel-JavaScript, JupyterKernel-Julia, JupyterKernel-Octave, JupyterKernel-PyDeepLearning, JupyterKernel-PyParaView, JupyterKernel-PyVisualization, JupyterKernel-PyQuantum, JupyterKernel-R, JupyterKernel-Ruby, JupyterProxy-Matlab, JupyterProxy-XpraHTML5, LAME, LMDB, LZO, LevelDB, LibTIFF, LibUUID, Libint, LittleCMS, M4, MATIO, MATLAB, NASM, NLopt, OPARI2, OTF2, OpenImageDenoise, PCRE, PDT, PROJ, Pango, Pmw, PnMPI, PyCairo, PyGObject, Python-Xpra, Qhull, Qt, Qt5, S-Lang, SCons, SIP, SQLite, SWIG, Serf, SoX, Szip, Tk, UDUNITS, XKeyboardConfig, XML-Parser, XZ, Yasm, Zip, Zip, adwaita-icon-theme, ant, assimp, babl, binutils, byacc, bzip2, cairo, cling, configurable-http-proxy, damageproto, dbus-glib, eudev, expat, fixesproto, flatbuffers, fontconfig, fontsproto, freeglut, freetype, g2clib, g2lib, gc, gettext, gexiv2, gflags, giflib, glog, glproto, googletest, gperf, gsettings-desktop-schemas, guile, gzip, icc, ifort, inputproto, intltool, itstool, jhbuild, jsc-xdg-menu, kbproto, libGLU, libICE, libSM, libX11, libXau, libXaw, libXcursor, libXdamage, libXdmcp, libXext, libXfixes, libXfont, libXft, libXi, libXinerama, libXmu, libXp, libXpm, libXrandr, libXrender, libXt, libXtst, libcerf, libcroco, libctl, libdap, libdrm, libdwarf, libelf, libepoxy, libevent, libffi, libfontenc, libgd, libgeotiff, libgit2, libglade, libiconv, libidn, libjpeg-turbo, libmatheval, libmypaint, libpciaccess, libpng, libpthread-stubs, libreadline, librsvg, libsndfile, libspatialindex, libspng, libtirpc, libtool, libunistring, libunwind, libwebp, libxcb, libxkbcommon, libxml2, libxml2-python, libxslt, libyaml, libyuv, lz4, makedepend, makeinfo, motif, msgpack-c, muparserx, ncurses, nettle, nlohmann-json, nodejs, nsync, nsync, nvenc_sdk, nvidia, nvidia-driver, nvidia-Video_Codec_SDK, openvkl, patchelf, pixman, pkg-config, pkgconfig, popt, printproto, protobuf, protobuf-python, pscom, pybind11, qrupdate, randrproto, recordproto, rencode, renderproto, rkcommon, scrollkeeper, snappy, spdlog, SZ, texinfo, uglifyjs, util-linux, vpx, wxPropertyGrid, wxWidgets, x264, x265, xbitmaps, xcb-proto, xcb-util, xcb-util-image, xcb-util-keysyms, xcb-util-renderutil, xcb-util-wm, xextproto, xineramaproto, xorg-macros, xprop, xproto, xtrans, yuicompressor, zfp, zlib, 
hide-toolchains               (E) = GCCcore
hooks                         (E) = /p/fastdata/zam/swmanage/EasyBuild/2022/Custom_Hooks/eb_hooks.py
include-easyblocks            (E) = /p/fastdata/zam/swmanage/EasyBuild/2022/Custom_EasyBlocks/*.py, /p/fastdata/zam/swmanage/EasyBuild/2022/Custom_EasyBlocks/generic/*.py
include-module-naming-schemes (E) = /p/fastdata/zam/swmanage/EasyBuild/2022/Custom_MNS/*.py
include-toolchains            (E) = /p/fastdata/zam/swmanage/EasyBuild/2022/Custom_Toolchains/*.py, /p/fastdata/zam/swmanage/EasyBuild/2022/Custom_Toolchains/fft/*.py, /p/fastdata/zam/swmanage/EasyBuild/2022/Custom_Toolchains/compiler/*.py
installpath                   (E) = /p/project/ccstao/cstao05/easybuild/juwelsbooster
job-backend                   (E) = Slurm
minimal-toolchains            (E) = True
module-naming-scheme          (E) = FlexibleCustomHierarchicalMNS
moduleclasses                 (E) = base, astro, bio, cae, chem, compiler, data, debugger, devel, geo, ide, lang, lib, math, mpi, numlib, perf, quantum, phys, system, toolchain, tools, vis, astro, base, bio, cae, chem, compiler, data, debugger, devel, geo, ide, lang, lib, math, mpi, numlib, perf, phys, quantum, sidecompiler, system, toolchain, tools, vis
optarch                       (E) = {'Intel': 'march=core-avx2'}
packagepath                   (E) = /p/project/ccstao/cstao05/easybuild/juwelsbooster/packages
parallel                      (E) = 8
prefix                        (E) = /p/project/ccstao/cstao05/easybuild/juwelsbooster
repositorypath                (E) = /p/project/ccstao/cstao05/easybuild/juwelsbooster/ebfiles_repo
robot                         (E) = /p/fastdata/zam/swmanage/EasyBuild/2022/Overlays/juwelsbooster_overlay, /p/fastdata/zam/swmanage/EasyBuild/2022/Golden_Repo, /p/project/ccstao/cstao05/easybuild/juwelsbooster/ebfiles_repo, /p/fastdata/zam/swmanage/EasyBuild/2022/Overlays/juwelsbooster_overlay, /p/fastdata/zam/swmanage/EasyBuild/2022/Golden_Repo
robot-paths                   (E) = /p/fastdata/zam/swmanage/EasyBuild/2022/Overlays/juwelsbooster_overlay, /p/fastdata/zam/swmanage/EasyBuild/2022/Golden_Repo, /p/project/ccstao/cstao05/easybuild/juwelsbooster/ebfiles_repo, /p/fastdata/zam/swmanage/EasyBuild/2022/Overlays/juwelsbooster_overlay, /p/fastdata/zam/swmanage/EasyBuild/2022/Golden_Repo
search-paths                  (E) = /p/software/juwelsbooster/stages/2022/software/EasyBuild/4.5.5/easybuild/easyconfigs
sourcepath                    (E) = /p/project/ccstao/cstao05/easybuild/juwelsbooster/sources
sticky-bit                    (E) = True
test-report-env-filter        (E) = re.compile('.*PS1.*|PROMPT.*|.*LICENSE.*|.*PROJECT.*|.*DATA.*|.*FASTDATA.*|.*SCRATCH.*|.*IMESCRATCH.*|.*HOME.*|.*ARCHIVE.*|.*LOGNAME.*|^SSH|USER|HOSTNAME|UID|.*COOKIE.*')
trace                         (E) = True
umask                         (E) = 022
use-existing-modules          (E) = True
rsarm commented 2 years ago

I'm finding the same issue with the PythonBundle easyblock. I tested with the Tarball easyblock, just to choose one, and that didn't happen.

branfosj commented 2 years ago

See the potential fix in #4063