Add in dwarf2json warning about sybmols with rust namespace

ikelos commented 1 month ago

This should raise a warning if loading a file produced by dwarf2json < 0.8.0. Unfortunately dwarf2json didn't include the kernel version in the metadata, so we have to warn on all symbols generated by dwarf2json < 0.8.0.

Can someone please check the specific version of linux where this started to be an issue (rust namespace symbols were included), and if possible can we update dwarf2json to include the version of the kernel in the metadata somewhere?

ikelos commented 1 month ago

It will go off a lot unless we can find a way to determine the kernel version from somewhere (and the filename isn't suitable, it needs to be data in the JSON). Best I've got would be to parse the kernel banner out of the banner key and then check it's version? 5:S

eve-mem commented 1 month ago

@ikelos if you did want to try parsing out the linux version then something like this works on my tests. Can then use linux_version later to choose to display the message or not.

# attempt to read linux version from linux_banner constant_data
    linux_banner = base64.b64decode(
        input.get("symbols", {}).get("linux_banner", {}).get("constant_data", None)
    )
    linux_version_regex = re.compile(rb"Linux version ([0-9]+)\.([0-9]+)\.([0-9]+)")
    regex_match = linux_version_regex.search(linux_banner)
    try:
        major = int(regex_match.group(1))
        minor = int(regex_match.group(2))
        patch = int(regex_match.group(3))
        linux_version = (major, minor, patch)
    except IndexError:
        major = None
        minor = None
        patch = None
        linux_version = None

Abyss-W4tcher commented 1 month ago

The only observed impacted vendor, for now, is Ubuntu. A brief summary :

linux-lib-rust-6.8.0-28-generic | linux-lib-rust-6.8.0-31-generic | linux-lib-rust-6.8.0-32-generic | linux-lib-rust-6.8.0-38-generic | linux-lib-rust-6.8.0-40-generic
linux-lib-rust-6.8.0-39-generic | linux-lib-rust-6.8.0-43-generic | linux-lib-rust-6.8.0-44-generic | linux-lib-rust-6.8.0-41-generic | linux-lib-rust-6.8.0-46-generic
linux-lib-rust-6.8.0-45-generic | linux-lib-rust-6.8.0-48-generic | linux-lib-rust-6.5.0-9-generic | linux-lib-rust-6.8.0-22-generic | linux-lib-rust-6.8.0-34-generic
linux-lib-rust-6.8.0-35-generic | linux-lib-rust-6.8.0-36-generic

Rust integration is up to the vendor, and we've only seen Ubuntu issues for now. So, a custom made kernel from 6.2.0 might embed rust bindings, but not a 6.10.0 mainline Debian one.

I might have a solution to determine if rust bindings are present, relying on symbols :

Kernel hardcoded rust helpers are exported through EXPORT_SYMBOL_GPL and will be present in the C symbols compile unit. Example from a "rust" ISF (before or after 0.9.0) :

        "rust_helper_BUG": {
            "type": {
                "kind": "base",
                "name": "void"
            },
            "address": 18446744071587506080
        },

rust_helper_BUG is the oldest exported helper symbol I could find, and should enhance compatibility.

So, if rust_helper_BUG is there and dwarf2json<0.9.0, it means we will likely encounter type confusion, which means vol3 needs to make a warning or a graceful exit.

I am still testing if this is reliable, to see if rust_helper_BUG will be here, independently if rust was implemented.

Abyss-W4tcher commented 1 month ago

Testing seems to strongly confirms the observation, as CONFIG_RUST needs to be toggled on, and rust helpers will be automatically imported and generated in the context. Meaning rust_helper_BUG shouldn't be present in non-rust flavored kernels.

A bit of doc. for the curious :

ikelos commented 1 month ago

Awesome, are you ok to fix up this PR (I'm not in the country at the moment so my ability to code is somewhat limited), if not, then no worries I can look into it on Thursday. Really nice find on the rust symbol though, that looks like the most reliable way to go! 5:)

ikelos commented 1 month ago

Superceded by #1315.

volatilityfoundation / volatility3

Add in dwarf2json warning about sybmols with rust namespace #1305