Creating Symbol Table for Monterey - Failed validating 'oneOf' in schema

[reggyledoux]

_./dwarf2json mac --macho /Library/Developer/KDKs/KDK_12.2.1_21D62.kdk/System/Library/Kernels/kernel.dSYM/Contents/Resources/DWARF/kernel --macho-symbols /Library/Developer/KDKs/KDK_12.2.1_21D62.kdk/System/Library/Kernels/kernel > monterey_Kernel12.2.121D62.json

_./dwarf2json mac --macho /Library/Developer/KDKs/KDK_12.2.1_21D62.kdk/System/Library/Kernels/kernel.release.t6000.dSYM/Contents/Resources/DWARF/kernel.release.t6000 --macho-symbols /Library/Developer/KDKs/KDK_12.2.1_21D62.kdk/System/Library/Kernels/kernel.release.t6000 > monterey_Kernel12.2.1_21D62t6000.json

Compressed json files to the xz format and copied to volatility3/symbols/mac

copied compressed file to volatility3/symbols/mac

Run volatility isfinfo to ensure the new symbol table is recognised

_python3 vol.py isfinfo | grep mac file:///volatility3/framework/symbols/mac/sierra_12.2.1_16G2128.json.xz True (cached) 18 0 43736 166 b'Darwin Kernel Version 16.7.0: Sun Jun 2 20:26:31 PDT 2019; root:xnu-3789.73.50~1/RELEASE_X86_64\x00' file:///volatility3/symbols/mac/monterey_Kernel12.2.1_21D62_t6000.json.xz True (cached) 19 0 58919 361 b'Darwin Kernel Version 21.3.0: Wed Jan 5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64_T6000\x00' file:///volatility3/symbols/mac/monterey_Kernel12.2.1_21D62.json.xz True (cached) 19 0 62951 369 b'Darwin Kernel Version 21.3.0: Wed Jan 5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_X8664\x00'

Python3 vol.py isfinfo | grep mac

Run the volatility3 banner on memory to ensure the symbol table is a match

python3 vol.py -f OSX_Monterey.lime banners Volatility 3 Framework 2.4.1 Progress: 100.00 PDB scanning finished
Offset Banner

_0x10004aa4bd5 Darwin Kernel Version 21.3.0: Wed Jan 5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64_T6000 0x10004aa4c3c Darwin Kernel Version 21.3.0: Wed Jan 5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64_T6000 0x10008516d8c Darwin Kernel Version 21.3.0: Wed Jan 5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64_T6000 0x1000e9b3100 Darwin Kernel Version 21.3.0: Wed Jan 5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64T6000

Finally, run mac.pslist plugin (with -vvvvv).

_DEBUG volatility3.framework.automagic.symbol_cache: Duplicate entry for identifier b'Darwin Kernel Version 16.7.0: Sun Jun 2 20:26:31 PDT 2019; root:xnu-3789.73.50~1/RELEASE_X86_64\x00': file://volatility3/volatility3/framework/symbols/mac/sierra_12.2.1_16G2127.json.xz and file:///volatility3/volatility3/framework/symbols/mac/sierra_12.2.1_16G2128.json.xz DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.3.0: Wed Jan 5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64_T6000\x00' DEBUG volatility3.schemas: Validating JSON against schema... Level 7 volatility3.framework.automagic.stacker: Exception during stacking: {'count': 33, 'kind': 'array', 'subtype': None} is not valid under any of the given schemas

Failed validating 'oneOf' in schema['properties']['user_types']['additionalProperties']['properties']['fields']['additionalProperties']['properties']['type']: {'oneOf': [{'$ref': '#/definitions/type_pointer'}, {'$ref': '#/definitions/type_base'}, {'$ref': '#/definitions/type_array'}, {'$ref': '#/definitions/type_struct'}, {'$ref': '#/definitions/type_enum'}, {'$ref': '#/definitions/type_function'}, {'$ref': '#/definitions/type_bitfield'}]}

On instance['user_types']['host']['fields']['special']['type']: {'count': 33, 'kind': 'array', 'subtype': None} Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsof.kernel.layer_name Level 9 volatility3.framework.configuration.requirements: TypeError - Layer is not the required Architecture: LimeLayer Level 9 volatility3.framework.configuration.requirements: TypeError - Layer is not the required Architecture: FileLayer DEBUG volatility3.framework.automagic.stacker: Stacked layers: ['LimeLayer', 'FileLayer'] INFO volatility3.framework.automagic: Running automagic: SymbolFinder
INFO volatility3.framework.automagic: Running automagic: MacSymbolFinder Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsof.kernel.symbol_table_name INFO volatility3.framework.automagic: Running automagic: KernelModule Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsof.kernel.layer_name Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsof.kernel.symbol_table_name Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsof.kernel.layer_name Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsof.kernel.layer_name Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsof.kernel.symbol_tablename

Any help here would be greatly appreciated

[ikelos]

Thanks very much for raising this (and thanks for having jsonschema installed and putting up with the verification time!). 5:D

This line seems to give the most indication:

Exception during stacking: {'count': 33, 'kind': 'array', 'subtype': None} is not valid under any of the given schemas

I'm going to transfer this bug over to the dwarf2json project. I think the issue is that it appears to have created an array object without specifying a type (even a void or void * type), which the schema validator then hasn't liked. For now you can either manually look for an array of size 33 and set the type explicitly to be something of the correct length, or wait for us to try and get the issue resolved in dwarf2json. 5:) We'll need to figure out why it doesn't emit a type, and whether that's because it doesn't know how to emit the right type, or if there really is no type information and we need to give it a placeholder of some kind...

[ikelos]

@ilch1 Any chance you could take a look at this please? @reggyledoux could you tell us which version of the KDK this was generated from so we can reproduce the output please?

[ilch1]

Sure, I’ll take a look.

---

From: ikelos @.> Sent: Friday, February 24, 2023 1:17:26 PM To: volatilityfoundation/dwarf2json @.> Cc: Ilya @.>; Mention @.> Subject: Re: [volatilityfoundation/dwarf2json] Creating Symbol Table for Monterey - Failed validating 'oneOf' in schema (Issue #46)

@ilch1https://github.com/ilch1 Any chance you could take a look at this please? @reggyledouxhttps://github.com/reggyledoux could you tell us which version of the KDK this was generated from so we can reproduce the output please?

— Reply to this email directly, view it on GitHubhttps://github.com/volatilityfoundation/dwarf2json/issues/46#issuecomment-1444184248, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABMXKRQZRFRYNLW4C5NKQXLWZD3LNANCNFSM6AAAAAAVHGHICE. You are receiving this because you were mentioned.Message ID: @.***>

[reggyledoux]

Many thanks for your reply and apologies for the delay!

_Kernel_Debug_Kit_12.2.121D62.dmg

output from sw_vers on mac where memory image was acquired.

ProductName: macOS ProductVersion: 12.2.1 BuildVersion: 21D62

[ilch1]

Hi @ikelos, @reggyledoux,

The monterey_Kernel12.2.1_21D62.json ISF created from x86 kernel appears valid and validated against volatility/schemas/schema-6.2.0.json schema.

The monterey_Kernel12.2.1_21D62_t6000.json ISF fails validation against volatility/schemas/schema-6.2.0.json schema (as you noted above). Examining that file, I see multiple types that have "subtype": null. I'll look into why that is.

There may be a workaround in the short-term. Could you please run sysctl kern.version and report the output on the machine on which OSX_Monterey.lime was acquired. Also, what tool/method did you use to acquire this lime sample?

Thanks!

[reggyledoux]

Thanks @ikelos

Here you go,

kern.version: Darwin Kernel Version 21.3.0: Wed Jan 5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64_T6000

Tools: Volexity Surge Collect Pro Release 23.01.23

[ilch1]

After discussing this with the volatility team, adding dwarf2json support has been postponed until Volatility3 can analyze macOS ARM memory samples.