RecursionError('maximum recursion depth exceeded')

Bug Description

Metasploit Framework builds are failing due to the following error:-

snapcraft internal error: RecursionError('maximum recursion depth exceeded')

To Reproduce

Launchpad builds are failing as can be seen in the build log attached below.

Environment

The snapcraft library linter fails to lint libraries of Metasploit Framework due to the above error.

snapcraft.yaml

name: metasploit-framework
base: core24
adopt-info: metasploit-framework
summary: The Metasploit Framework
description: |
  The **Metasploit Framework** (MSF) is far more than just a collection of exploits–it is also a solid foundation that you can build upon and easily customize to meet your needs. This allows you to concentrate on your unique target environment and not have to reinvent the wheel. MSF is considered to be one of the single most useful security auditing tools freely available to security professionals today. From a wide array of commercial grade exploits and an extensive exploit development environment, all the way to network information gathering tools and web vulnerability plugins, the Metasploit Framework provides a truly impressive work environment.

  Installation: `snap install metasploit-framework`

  How to use: https://www.offensive-security.com/metasploit-unleashed/

  Issues: https://github.com/JitPatro/metasploit-framework/issues

grade: stable
confinement: strict

platforms:
  amd64:
    build-on: [amd64]
  arm64:
    build-on: [arm64]
  armhf:
    build-on: [armhf]

environment:
  LANG: C.UTF-8
  LC_ALL: C.UTF-8
  PATH: "$SNAP/opt/metasploit-framework/bin:$SNAP/opt/metasploit-framework/embedded/bin:$SNAP/usr/sbin:$SNAP/usr/bin:$SNAP/sbin:$SNAP/bin:$PATH"
  LD_LIBRARY_PATH: "${LD_LIBRARY_PATH:+$LD_LIBRARY_PATH:}$SNAP/lib:$SNAP/usr/lib:$SNAP/lib/$CRAFT_ARCH_TRIPLET_BUILD_FOR:$SNAP/usr/lib/$CRAFT_ARCH_TRIPLET_BUILD_FOR:$SNAP/opt/metasploit-framework/embedded/lib"
  TERMINFO: $SNAP/opt/metasploit-framework/embedded/share/terminfo
  EDITOR: vi
  HOME: $SNAP_USER_COMMON
  TEMPDIR: $SNAP_USER_COMMON
  TMPDIR: $SNAP_USER_COMMON
  TEMP: $SNAP_USER_COMMON
  TMP: $SNAP_USER_COMMON

slots:
  metasploit:
    interface: content
    content: executables
    source:
      read:
        - $SNAP/opt

apps:
  msfconsole:
    command: opt/metasploit-framework/bin/msfconsole
    plugs: [home, network, network-bind, network-control, desktop]

  msfbinscan:
    command: opt/metasploit-framework/bin/msfbinscan
    plugs: [home]

  msfelfscan:
    command: opt/metasploit-framework/bin/msfelfscan
    plugs: [home]

  msfmachscan:
    command: opt/metasploit-framework/bin/msfmachscan
    plugs: [home]

  msfpescan:
    command: opt/metasploit-framework/bin/msfpescan
    plugs: [home]

  msfrop:
    command: opt/metasploit-framework/bin/msfrop
    plugs: [home]

  msfvenom:
    command: opt/metasploit-framework/bin/msfvenom
    plugs: [home, network]

  msfrpc:
    command: opt/metasploit-framework/bin/msfrpc
    plugs: [network]

  msfrpcd:
    command: opt/metasploit-framework/bin/msfrpcd
    plugs: [home, network, network-bind]

  msfd:
    command: opt/metasploit-framework/bin/msfd
    plugs: [network, network-bind]

  msfdb:
    command: opt/metasploit-framework/bin/msfdb
    plugs: [network, network-bind, home]

  msfupdate:
    command: opt/metasploit-framework/bin/msfupdate

package-repositories:
  - type: apt
    formats: [deb]
    suites: [lucid]
    components: [main]
    key-id: 09E55FAF4F7862CD6D558997CDFB5FA52007B954
    url: http://downloads.metasploit.com/data/releases/metasploit-framework/apt
    architectures: [amd64, arm64, armhf]

parts:

  nmap:
    source: https://nmap.org/dist/nmap-7.95.tgz
    plugin: make
    build-packages: [automake, build-essential, libssl-dev, python3-setuptools, upx-ucl]

    override-build: |
      if [ `arch` != "armv7l" ]
      then
        ./configure --with-libpcap=included --with-libpcre=included --without-nping --without-ncat --without-zenmap --prefix=${CRAFT_PART_INSTALL}
        sed -i 's/OPENSSL_LIBS = -lssl -lcrypto/OPENSSL_LIBS = -lssl -lcrypto -lpthread/g' ./Makefile
        make static
        make install
        cd ${CRAFT_PART_INSTALL}/bin; strip ./nmap; upx -9 ./nmap
      else
        echo "Build fails on armhf. Skipping..."
      fi

  metasploit-framework:
    plugin: nil
    stage-packages: [metasploit-framework, libfuse2t64, libncursesw6, libtinfo6, busybox]
    override-stage: |
      craftctl default
      sed -i 's#SCRIPTDIR=/opt#SCRIPTDIR=$SNAP/opt#g' ${CRAFT_STAGE}/opt/metasploit-framework/bin/*
      sed -i 's#INSTALL_DIR="/opt#INSTALL_DIR="$SNAP/opt#g' ${CRAFT_STAGE}/opt/metasploit-framework/bin/msfdb
      sed -i "s/'google-chrome'/'xdg-open', 'google-chrome'/g" ${CRAFT_STAGE}/opt/metasploit-framework/embedded/lib/ruby/gems/3.*/gems/rex-core-*/lib/rex/compat.rb
      sed -i "s#/opt/metasploit-framework/embedded/bin/ruby#/usr/bin/env ruby#g" ${CRAFT_STAGE}/opt/metasploit-framework/embedded/bin/*
      sed -i '/DEVELOPER_TIPS = \[/,+5d' ${CRAFT_STAGE}/opt/metasploit-framework/embedded/framework/lib/msf/ui/tip.rb
      sed -i 's/ALL_TIPS = COMMON_TIPS + DEVELOPER_TIPS/ALL_TIPS = COMMON_TIPS/g' ${CRAFT_STAGE}/opt/metasploit-framework/embedded/framework/lib/msf/ui/tip.rb
      sed -i 's/#{$0}/msfvenom/g' ${CRAFT_STAGE}/opt/metasploit-framework/embedded/framework/msfvenom
      echo -e '#!/bin/sh\n\nset -e\n\necho "msfupdate is no longer supported when Metasploit is installed as a snap package.\nPlease use:- snap refresh metasploit-framework"' > ${CRAFT_STAGE}/opt/metasploit-framework/bin/msfupdate
      craftctl set version=$(cd ../parts/metasploit-framework/stage_packages/ && ls metasploit* | awk -F_ '{ printf $2 }' | awk -F~ '{ printf "v"$1"-dev" }')

    override-prime: |
      craftctl default
      mkdir -p ${CRAFT_PRIME}/bin
      cd ${CRAFT_PRIME}/usr/bin; for binary in {awk,cat,chmod,chown,clear,cp,cut,date,diff,echo,find,grep,head,hexdump,ip,ls,mkdir,mv,netstat,nslookup,ps,pwd,rm,sed,sleep,sort,tail,tar,tee,touch,tr,vi,wc,wget,which,whoami,xxd}; do ln -sf busybox $binary; done

Relevant log output

The complete build log can be seen here.

Packing...
Reading snap metadata...
Running linters...
Running linter: classic
Running linter: library
snapcraft internal error: RecursionError('maximum recursion depth exceeded')
Full execution log: '/root/.local/state/snapcraft/log/snapcraft-20240803-233531.507594.log'
Build failed
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/lpbuildd/target/build_snap.py", line 324, in run
    self.build()
  File "/usr/lib/python3/dist-packages/lpbuildd/target/build_snap.py", line 308, in build
    self.run_build_command(["snapcraft"], cwd=output_path, env=env)
  File "/usr/lib/python3/dist-packages/lpbuildd/target/operation.py", line 62, in run_build_command
    return self.backend.run(args, cwd=cwd, env=full_env, **kwargs)
  File "/usr/lib/python3/dist-packages/lpbuildd/target/lxd.py", line 718, in run
    subprocess.check_call(cmd, **kwargs)
  File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)

Additional context

No response

Thanks for the report! I can reproduce this with pyelftools.

Reproducer with pyelftools

1.Build the snap to create the problematic binary (or use the base64 encoded copy of it below)

snapcraft

Grab the binary

lxc --project snapcraft file pull <name of lxc container>/root/prime/opt/metasploit-framework/embedded/framework/data/templates/template_x86_linux_dll.bin

run pyelftools

>>> with pathlib.Path("template_x86_linux_dll.bin").open("rb") as file:
...   elf_file = elffile.ELFFile(file)
...   for section in elf_file.iter_sections():
...     pass
...

Error

  File "<stdin>", line 3, in <module>
  File "/home/developer/dev/snapcraft/.venv/lib/python3.12/site-packages/e
lftools/elf/elffile.py", line 174, in iter_sections
    section = self.get_section(i)
              ^^^^^^^^^^^^^^^^^^^
  File "/home/developer/dev/snapcraft/.venv/lib/python3.12/site-packages/e
lftools/elf/elffile.py", line 141, in get_section
    return self._make_section(section_header)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/developer/dev/snapcraft/.venv/lib/python3.12/site-packages/e
lftools/elf/elffile.py", line 659, in _make_section
    return DynamicSection(section_header, name, self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/developer/dev/snapcraft/.venv/lib/python3.12/site-packages/e
lftools/elf/dynamic.py", line 237, in __init__
    stringtable = elffile.get_section(header['sh_link'])
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...
  File "/home/developer/dev/snapcraft/.venv/lib/python3.12/site-packages/e
lftools/elf/elffile.py", line 619, in _get_section_header
    return struct_parse(
           ^^^^^^^^^^^^^
  File "/home/developer/dev/snapcraft/.venv/lib/python3.12/site-packages/e
lftools/common/utils.py", line 43, in struct_parse
    return struct.parse_stream(stream)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/developer/dev/snapcraft/.venv/lib/python3.12/site-packages/e
lftools/construct/core.py", line 190, in parse_stream
    return self._parse(stream, Container())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/developer/dev/snapcraft/.venv/lib/python3.12/site-packages/e
lftools/construct/core.py", line 635, in _parse
    if "<obj>" in context:
       ^^^^^^^^^^^^^^^^^^
RecursionError: maximum recursion depth exceeded

Binary

> base64 template_x86_linux_dll.bin
f0VMRgEBAQAAAAAAAAAAAAMAAwABAAAA9gAAADQAAAB0AAAAAAAAADQAIAACACgAAgABAAEAAAAA
AAAAAAAAAAAAAADvvq3e776t3gcAAAAAEAAAAgAAAAcAAADEAAAAxAAAAMQAAAAwAAAAMAAAAAAQ
AAABAAAABgAAAAAAAADEAAAAxAAAADAAAAAAAAAAAAAAAAgAAAAHAAAAAAAAAAMAAAAAAAAA9AAA
APQAAAACAAAAAAAAAAAAAAAAAAAAAAAAAAwAAAD2AAAABQAAAPQAAAAGAAAA9AAAAAoAAAAAAAAA
CwAAAAAAAAAAAAAAAAAAAAAA

Workaround

As a workaround, you should be able to disable the linter.

lint:
  ignore:
    - library

Solution

I see a bug upstream, it looks like it's about to celebrate its 10 year anniversary: https://github.com/eliben/pyelftools/issues/60

Unfortunately, ignoring the troublesome file from the linter does not fix the problem.

Snapcraft needs to:

Ignore files when collecting all files here
Catch recursion errors and recommend that users inspect the file or ignore it when linting
Submit an upstream patch to pyelftools

canonical / snapcraft