Wizard of automagic AKA autoba(h)n-ner

[garanews]

If plugin banner that returns kernel version of linux/mac does't match the kernels contained in symbols, try to download the kernel source and create the correct symbol

[garanews]

[garanews]

Global

• [x] if not able to find automatically what download, provide URL
• [x] if not able to find automatically what download, upload package (deb, ddeb, rpm,..?)
• [x] upload direcly symbol (json, json.xz)
• [x] proxy support :(

Specific

• [x] Ubuntu

  • [x] find best matching repo
    • http://ddebs.ubuntu.com/ubuntu/pool/main/l/linux/
  • [x] steps to build symbols
    • download linux-image-xxxx-generic-dbgsym.ddeb
    • extract ddeb
    • extract data.tgz
    • find vmlinuz and pass it to dwarf
  • [x] identify regexp to match banner result with online link
    • banner: Linux version 5.8.0-25-generic (buildd@lcy01-amd64-022) (gcc (Ubuntu 10.2.0-13ubuntu1) 10.2.0, GNU ld (GNU Binutils for Ubuntu) 2.35.1) #26-Ubuntu SMP Thu Oct 15 10:30:38 UTC 2020 (Ubuntu 5.8.0-25.26-generic 5.8.14)
    • online package: linux-image-unsigned-5.8.0-25-generic-dbgsym_5.8.0-25.26_amd64.ddeb
  • [x] integrate in OROCHI

• [x] Debian

  • [x] find best matching repo
    • https://deb.sipwise.com/debian/pool/main/l/linux/
  • [x] steps to build symbols
    • download linux-image-xxxxx-dbg_xxxxxx.deb
    • extract deb
    • extract data.tgz
    • find vmlinuz and pass it to dwarf
  • [x] identify regexp to match banner result with online link
    • banner: Linux version 4.9.0-8-amd64 (debian-kernel@lists.debian.org) (gcc version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1) ) #1 SMP Debian 4.9.130-2 (2018-10-27)
    • online package: linux-image-4.9.0-8-amd64-dbg_4.9.130-2_amd64.deb
  • [x] integrate in OROCHI

• [ ] RedHat

  • [x] find best matching repo
    • https://access.redhat.com/downloads/content/package-browser but need credentials
  • [x] steps to build symbols
    • extract rpm
    • find vmlinuz and pass it to dwarf
  • [ ] identify regexp to match banner result with online link
    • banner: Linux version 4.18.0-240.15.1.el8_3.x86_64 (mockbuild@x86-vm-07.build.eng.bos.redhat.com) (gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)) #1 SMP Wed Feb 3 03:12:15 EST 2021
    • online package: kernel-debuginfo-4.18.0-240.15.1.el8_3.x86_64.rpm
  • [ ] integrate in OROCHI

• [ ] Fedora

  • [x] find best matching repo
    • https://archives.fedoraproject.org/pub/fedora/linux/releases/
  • [x] steps to build symbols
    • extract rpm
    • find vmlinuz and pass it to dwarf
  • [ ] identify regexp to match banner result with online link
    • banner: Linux version 5.8.15-301.fc33.x86_64 (mockbuild@bkernel01.iad2.fedoraproject.org) (gcc (GCC) 10.2.1 20200826 (Red Hat 10.2.1-3), GNU ld version 2.35-10.fc33) #1 SMP Thu Oct 15 16:58:06 UTC 2020
    • online package: kernel-debuginfo-5.8.15-301.fc33.x86_64.rpm
  • [ ] integrate in OROCHI

• [ ] other distros

[dadokkio]

django_1     | 172.21.0.1:40816 - - [16/Jan/2024:15:44:08] "GET /symbols?index=dd17ccfa-b485-11ee-890e-0242ac150005" 200 4291
django_1     |  - Downloading https://deb.sipwise.com/debian/pool/main/l/linux/linux-image-4.19.0-5-amd64-dbg_4.19.37-5_amd64.deb
django_1     |  - Extracting ./usr/lib/debug/lib/modules/4.19.0-5-amd64/vmlinux
django_1     |  - Writing to /tmp/vmlinuxwg0426e3
django_1     | Processing Files...
django_1     |  - Running ['/dwarf2json/./dwarf2json', 'linux', '--elf', '/tmp/vmlinuxwg0426e3']
django_1     |  - Writing to /src/volatility3/volatility3/symbols/linux/added_4.19.0-5-amd64-dbg_4.19.37-5_amd64.json.xz
django_1     | Done
django_1     | ERROR 2024-01-16 15:52:20,257 log 36 140467215779584 Internal Server Error: /symbols
django_1     | Traceback (most recent call last):
django_1     |   File "/usr/local/lib/python3.11/site-packages/asgiref/sync.py", line 534, in thread_handler
django_1     |     raise exc_info[1]
django_1     |   File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 42, in inner
django_1     |     response = await get_response(request)
django_1     |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
django_1     |   File "/usr/local/lib/python3.11/site-packages/asgiref/sync.py", line 534, in thread_handler
django_1     |     raise exc_info[1]
django_1     |   File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 253, in _get_response_async
django_1     |     response = await wrapped_callback(
django_1     |                ^^^^^^^^^^^^^^^^^^^^^^^
django_1     |   File "/usr/local/lib/python3.11/site-packages/asgiref/sync.py", line 479, in __call__
django_1     |     ret: _R = await loop.run_in_executor(
django_1     |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
django_1     |   File "/usr/local/lib/python3.11/site-packages/asgiref/current_thread_executor.py", line 40, in run
django_1     |     result = self.fn(*self.args, **self.kwargs)
django_1     |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django_1     |   File "/usr/local/lib/python3.11/site-packages/asgiref/sync.py", line 538, in thread_handler
django_1     |     return func(*args, **kwargs)
django_1     |            ^^^^^^^^^^^^^^^^^^^^^
django_1     |   File "/usr/local/lib/python3.11/contextlib.py", line 81, in inner
django_1     |     return func(*args, **kwds)
django_1     |            ^^^^^^^^^^^^^^^^^^^
django_1     |   File "/usr/local/lib/python3.11/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapper_view
django_1     |     return view_func(request, *args, **kwargs)
django_1     |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django_1     |   File "/app/orochi/website/views.py", line 1319, in symbols
django_1     |     if check_runnable(dump.pk, dump.operating_system, dump.banner):
django_1     |        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django_1     |   File "/app/orochi/utils/volatility_dask_elk.py", line 756, in check_runnable
django_1     |     if banners := automagic.linux.LinuxSymbolFinder(ctx, "").banners:
django_1     |                   ^^^^^^^^^^^^^^^
django_1     | AttributeError: module 'volatility3.framework.automagic' has no attribute 'linux'

need to be fixed for new automagic + cache logic