Signature

C++23 binary signature search algorithm optimized with AVX2 and TBB (optional).

• Single header library.
• Supports disabling exceptions and RTTI.
• 23 times as fast, as using <algorithm>.

Compiler Support

This library was benchmarked and tested using the following compilers.

• VS 2022 (cl, clang, clang-cl) on Windows
• LLVM 15 (clang and libc++) on Linux

Usage

Copy signature.hpp to your include path or install this project as an interface library using CMake.

#include <qis/signature.hpp>
#include <vector>
#include <cassert>
#include <cstdint>
#include <cstring>

// Optional: Make sure that AVX2 support was detected.
// static_assert(QIS_SIGNATURE_USE_AVX2);

// Optional: Make sure that TBB headers were found.
// static_assert(QIS_SIGNATURE_USE_TBB);

int main() {
  // Create a 1 GiB memory block.
  std::vector<std::uint8_t> memory;
  memory.resize(1024 * 1024 * 1024, 0);

  // Write data to the memory block.
  std::memcpy(memory.data() + memory.size() - 5, "\x01\xCD\xE3\x04\x05", 5);

  // Create signature.
  const qis::signature search("01 ?? E? 04");

  // Scan memory block.
  const auto pos = qis::scan(memory.data(), memory.size(), search);

  // Check that the signature was found.
  assert(pos != qis::npos);

  // Check data position.
  assert(pos == memory.size() - 5);
}

Use qis/xorstr to hide the signature string from analysis tools if necessary.

Tests

The container and search algorithms were extensively tested. See tests.hpp for more details.

Benchmarks

All benchmark results use the following report syntax.

                      + Scanned memory block size.
                      v
-----------------------------------------------------------------
Type |  K | AVX TBB | 1 GiB     Time             CPU   Iterations
-----------------------------------------------------------------
mask | 12 |         |   853445625 ns    853515625 ns            8
mask | 12 |     tbb |   129104875 ns    123046875 ns            8
mask | 12 | avx     |    50333100 ns     46875000 ns            8
mask | 12 | avx tbb |    36736000 ns     37109375 ns            8
 ^      ^    ^   ^
 |      |    |   +----- Benchmark compiled with TBB support.
 |      |    +--------- Benchmark compiled with AVX2 support.
 |      +-------------- Signature size used for the scan.
 +--------------------- Signature type used for the scan.

There are two types of signatures used for the benchmarks:

• data has all bytes defined and the search algorithm is optimized for it.

DB E7 DB DA EB DE E1 EC DD DE DD D6 EF E7 D7 EF E7 EA E7 DF D2 D7 E6 D6 D0 D9

• mask contains an unmasked 4-bit pattern (?) and uses a slower algorithm.

DB E? DB DA EB DE E1 EC DD DE DD D6 EF E7 D7 EF E7 EA E7 DF D2 D7 E6 D6 D0 D9

The K column shows how many bytes of the signature type were used for the scan.
In the example above, mask | 12 means DB E? DB DA EB DE E1 EC DD DE DD D6.

The scanned memory block always contains the signature at the very end.

All benchmark results were recorded on the same system.

Intel(R) Core(TM) i7-8700K @ 3.70 GHz (12 Threads)
* L1 Data 32 KiB (x6)
* L1 Instruction 32 KiB (x6)
* L2 Unified 256 KiB (x6)
* L3 Unified 12288 KiB (x1)

32 GB DDR4 RAM @ 2133 MHz (Dual Channel)

• See windows.md for Windows results.
• See linux.md for Linux results.

Tools

Install tools required for building benchmarks and tests.

Build & Install

This is a header-only. Building and installing is optional.