Add machometa unit

[cxiao]

As per the discussion in #35, this adds a new unit, machometa, which extracts similar metadata from Mach-O files as the existing pemeta unit.

$ machometa -h
usage: machometa [-h] [-L] [-Q] [-0] [-v] [-c] [-H] [-K] [-S] [-V] [-D] [-E] [-I] [-t]

Extract metadata from Mach-O files.

options:
  -c, --custom         Unless enabled, all default categories will be extracted.
  -H, --header         Parse basic data from the Mach-O header.
  -K, --linked-images  Parse all library images linked by the Mach-O.
  -S, --signatures     Parse signature and entitlement information.
  -V, --version        Parse version information from the Mach-O load commands.
  -D, --load-commands  Parse load commands from the Mach-O header.
  -E, --exports        List all exported functions.
  -I, --imports        List all imported functions.
  -t, --tabular        Print information in a table rather than as JSON

generic options:
  -h, --help           Show this help message and exit.
  -L, --lenient        Allow partial results as output.
  -Q, --quiet          Disables all log output.
  -0, --devnull        Do not produce any output.
  -v, --verbose        Specify up to two times to increase log level.

Example output

``` $ ef a64fa9f1c76457ecc58402142a8728ce34ccba378c17318b3340083eeb7acc67 | machometa { "FileType": "FAT", "Slices": [ { "Header": { "type": "mach_header_64", "magic": 4277009103, "cputype": "X86_64", "cpusubtype": "ALL", "filetype": "DYLIB", "loadcount": 14, "loadsize": 2312, "flags": [ "NOUNDEFS", "DYLDLINK", "TWOLEVEL", "NO_REEXPORTED_DYLIBS" ], "reserved": 0 }, "Linked Images": { "LOAD_DYLIB": [ "/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation", "/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation", "/usr/lib/libobjc.A.dylib", "/usr/lib/libSystem.B.dylib" ] }, "Signatures": { "Ad-Hoc Signed": false, "Signature Identifier": "libffmpeg", "Signature": { "Timestamp": "2023-03-13 06:41:00+00:00", "TimestampIssuer": "Developer ID Certification Authority", "Subject": "Developer ID Application: 3CX (33CF4654HL)", "SubjectLocation": "US", "ValidFrom": "2019-04-11 12:03:36+00:00", "ValidUntil": "2024-04-11 12:03:36+00:00", "Issuer": "Developer ID Certification Authority", "Fingerprint": "7df5ed6d71b296ed073a5b3efbcdc4c916ba41be", "Serial": "4b0aaf622b260469" }, "Requirements": "000000010000000300000014fade0c0000000098000000010000000600000002000000096c696266666d706567000000000000060000000f000000060000000e000000010000000a2a864886f76364060206000000000000000000060000000e000000000000000a2a864886f7636406010d0000000000000000000b000000000000000a7375626a6563742e4f550000000000010000000a3333434634363534484c0000", "Entitlements": "\n\n\n \n com.apple.security.cs.allow-jit\n \n com.apple.security.cs.allow-unsigned-executable-memory\n \n com.apple.security.cs.debugger\n \n com.apple.security.device.audio-input\n \n com.apple.security.device.camera\n \n \n\n" }, "UUID": "4c4c445f55553144a1e550e749c861fd", "Base Name": "libffmpeg.dylib", "Install Name": "@loader_path/libffmpeg.dylib" }, { "Header": { "type": "mach_header_64", "magic": 4277009103, "cputype": "ARM64", "cpusubtype": "ALL", "filetype": "DYLIB", "loadcount": 12, "loadsize": 1656, "flags": [ "NOUNDEFS", "DYLDLINK", "TWOLEVEL", "NO_REEXPORTED_DYLIBS" ], "reserved": 0 }, "Linked Images": { "LOAD_DYLIB": [ "/usr/lib/libSystem.B.dylib" ] }, "Signatures": { "Ad-Hoc Signed": false, "Signature Identifier": "libffmpeg", "Signature": { "Timestamp": "2023-03-13 06:41:01+00:00", "TimestampIssuer": "Developer ID Certification Authority", "Subject": "Developer ID Application: 3CX (33CF4654HL)", "SubjectLocation": "US", "ValidFrom": "2019-04-11 12:03:36+00:00", "ValidUntil": "2024-04-11 12:03:36+00:00", "Issuer": "Developer ID Certification Authority", "Fingerprint": "7df5ed6d71b296ed073a5b3efbcdc4c916ba41be", "Serial": "4b0aaf622b260469" }, "Requirements": "000000010000000300000014fade0c0000000098000000010000000600000002000000096c696266666d706567000000000000060000000f000000060000000e000000010000000a2a864886f76364060206000000000000000000060000000e000000000000000a2a864886f7636406010d0000000000000000000b000000000000000a7375626a6563742e4f550000000000010000000a3333434634363534484c0000", "Entitlements": "\n\n\n \n com.apple.security.cs.allow-jit\n \n com.apple.security.cs.allow-unsigned-executable-memory\n \n com.apple.security.cs.debugger\n \n com.apple.security.device.audio-input\n \n com.apple.security.device.camera\n \n \n\n" }, "Version": { "BuildVersion": { "Platform": "MACOS", "MinOS": "11.0.0", "SDK": "12.3.0", "Ntools": 1 } }, "UUID": "4c4c449f55553144a16d1b7f53c45f28", "Base Name": "libffmpeg.dylib", "Install Name": "@rpath/libffmpeg.dylib" } ] } ```

This also adds the k2l library as a dependency to Refinery.

New unit tests have been added in test/units/formats/macho/test_machometa.py. The following samples, which are used in the unit tests, have been uploaded to Malshare:

SHA-256	Description	Reference
6c121f2b2efa6592c2c22b29218157ec9e63f385e7a1d7425857d603ddef8c59	UpdateAgent binary from North Korean supply chain compromise of 3CX software	https://objective-see.org/blog/blog_0x74.html (referred to by SHA-1 hash 9e9a5f8d86356796162cee881c843cde9eaedfb3)
3e4bbd21756ae30c24ff7d6942656be024139f8180b7bddd4e5c62a9dfbd8c79	ARM64 MacOS version of LockBit Ransomware	https://objective-see.org/blog/blog_0x75.html (referred to by SHA-1 hash 2d15286d25f0e0938823dcd742bc928e78199b3d)
1a9a5c797777f37463b44de2b49a7f95abca786db3977dcdac0f79da739c08ac	SysJoker MacOS backdoor	https://intezer.com/blog/incident-response/new-backdoor-sysjoker/
a64fa9f1c76457ecc58402142a8728ce34ccba378c17318b3340083eeb7acc67	libffmpeg binary from North Korean supply chain compromise of 3CX software	https://objective-see.org/blog/blog_0x73.html (referred to by SHA-1 hash 769383fc65d1386dd141c960c9970114547da0c2
38c9b858c32fcc6b484272a182ae6e7f911dea53a486396037d8f7956d2110be	tasker binary component of SilverSparrow	https://redcanary.com/blog/clipping-silver-sparrows-wings/ (referred to by MD5 hash b370191228fef82635e39a137be470af)

Please feel free to directly make edits as needed.

[cxiao]

@huettenhain It looks like unit tests are failing due to a missing Malshare API key. Is this expected for unit tests running via GitHub actions? Should I submit a PR against https://github.com/binref/refinery-test-data with the samples instead?

[huettenhain]

It should work without the Malshare API key, that's just a fallback. However, GitHub has recently been a little flaky and I had tests failing before. I'll figure out what's wrong.

[huettenhain]

I was a little slow there to understand what's going on. Yes, the samples need to be in that repository, but I can add them myself.

[huettenhain]

Also: This is probably the most pristine PR I have ever seen. 🙇

[codecov[bot]]

Codecov Report

Merging #36 (d4c62fc) into master (24f5ef9) will decrease coverage by 0.15%. Report is 20 commits behind head on master. The diff coverage is 86.14%.

@@            Coverage Diff             @@
##           master      #36      +/-   ##
==========================================
- Coverage   83.76%   83.61%   -0.15%     
==========================================
  Files         332      336       +4     
  Lines       25754    26071     +317     
==========================================
+ Hits        21572    21799     +227     
- Misses       4182     4272      +90     

Files Changed	Coverage Δ
refinery/units/formats/macho/machometa.py	86.14% <86.14%> (ø)

... and 14 files with indirect coverage changes

[huettenhain]

Merged this with rebase; thank you very much for your contribution!