brexhq / substation

Substation is a toolkit for routing, normalizing, and enriching security event and audit logs.
https://substation.readme.io
MIT License
330 stars 21 forks source link

feat: Add Benchmark App & No-Op Features #108

Closed jshlbrd closed 1 year ago

jshlbrd commented 1 year ago

Description

Motivation and Context

The no-op features are helpful for development and integration testing (e.g., deploy a Substation node that doesn't output data but still processes it), and the benchmark app provides an easy way to do application-level performance tests. Here are some examples:

256 MB memory, 1 CPU

$ ./cmd/development/benchmark/benchmark -config examples/quickstart/config.json -input examples/quickstart/data.json 
100000 events in 7.571924086s (13206.68 events/sec)
1 MB in 7.571924086s (0.16 MB/sec)

1 GB memory, 2 CPU

$ ./cmd/development/benchmark/benchmark -config examples/quickstart/config.json -input examples/quickstart/data.json 
100000 events in 865.146917ms (115587.30 events/sec)
1 MB in 865.146917ms (1.43 MB/sec)

2 GB memory, 4 CPU

$ ./cmd/development/benchmark/benchmark -config examples/quickstart/config.json -input examples/quickstart/data.json 
100000 events in 947.320584ms (105560.89 events/sec)
1 MB in 947.320584ms (1.31 MB/sec)

How Has This Been Tested?

Tested locally, no new unit tests required.

Types of changes

Checklist: