search

smartcat-labs / berserker

Berserker is load generator with pluggable input source and configurable output.
Apache License 2.0
52 stars 8 forks source link

readme

Berserker

Load generator with modular architecture.

Build Status Download

Introduction

Berserker is designed to be modular from beginning as illustrated on the following diagram.

Core Design

Rate generator controls the rate at which load generator operates, rate is expressed on per second basis, or better say, number of impulses which will be generated within one second. Each time impulse is generated load generator fetches data from data source and passes it to worker. Since those are simple interfaces, it is easy to add additional module implementing either data source, worker and even rate generator. Following diagram represents possible modules for Load Generator of which some are already implemented.

Architecture

Berserker is designed as command line tool, but having modular architecture makes it easy to use it as Java library as well.

Berserker Commons

Berserker Commons holds interface for core and configuration and it provides signature all the modules need to confront to be able to work together.

Berserker Core

Berserker Core contains load generator implementation, and common implementations of data source, rate generator and worker.

Berserker Runner

Berserker Runner represents runnable jar where desired data source, rate generator and worker can be specified within YAML configuration. Following section illustrates YAML configuration example.

load-generator-configuration:
  data-source-configuration-name: Ranger
  rate-generator-configuration-name: default
  worker-configuration-name: Cassandra
  metrics-reporter-configuration-name: JMX
  thread-count: 10
  queue-capacity: 100000

data-source-configuration:
  values:
    id: uuid()
    firstName: random(['Peter', 'Mike', 'Steven', 'Joshua', 'John', 'Brandon'])
    lastName: random(['Smith', 'Johnson', 'Williams', 'Davis', 'Jackson', 'White', 'Lewis', 'Clark'])
    age: random(20..45)
    email: string('{}@domain.com', randomLengthString(5))
    statement:
      consistencyLevel: ONE
      query: string("INSERT INTO person (id, first_name, last_name, age, email) VALUES ({}, '{}', '{}', {}, '{}');", $id, $firstName, $lastName, $age, $email)
  output: $statement

rate-generator-configuration:
  rates:
    r: 1000
  output: $r

worker-configuration:
  connection-points: 0.0.0.0:32770
  keyspace: my_keyspace
  async: false
  bootstrap-commands:
    - "CREATE KEYSPACE IF NOT EXISTS my_keyspace WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1};"
    - USE my_keyspace;
    - CREATE TABLE IF NOT EXISTS person (id uuid, first_name text, last_name text, age int, email text, primary key (id));

metrics-reporter-configuration:
  domain: berserker
  filter:

Main part of configuration is load-generator-configuration where concrete modules which will be used for data source, rate generator and worker need to be specified. After load-generator-configuration section, there should be exactly one section for data source, rate generator and worker. Each section is allowed to contain module specific configuration as configuration interpretation will be done by module itself. In order for berserker-runner to be able to find particular module, each module jar must be in classpath.

Rate generator configuration

Documentation on rate generator configuration can be found here.

Modules

List of existing modules:

Berserker Ranger

Berserker Ranger is Ranger data source implementation.

Berserker Kafka

Berserker Kafka is worker implementation which sends messages to Kafka cluster.

Berserker Cassandra

Berserker Cassandra is worker implementation which executes CQL statements on Cassandra cluster.

Berserker HTTP

Berserker HTTP is worker implementation which sends HTTP request on configured endpoint.

Berserker RabbitMQ

Berserker RabbitMQ is worker implementation which sends AMQP messages to RabbitMQ.

Berserker MQTT

Berserker MQTT is worker implementation which publishes messages to MQTT broker.

Usage

Berserker can be used either as a library or as a stand-alone command line tool. Configuration examples can be used as starting point.

Library usage

Artifact can be fetched from bintray.

Add following repository element to your <repositories> section in pom.xml:

<repository>
  <id>bintray-smartcat-labs-maven</id>
  <name>bintray</name>
  <url>https://dl.bintray.com/smartcat-labs/maven</url>
</repository>

Add the dependency element to your <dependencies> section in pom.xml depending which artifact and version you need:

<dependency>
  <groupId>io.smartcat</groupId>
  <artifactId>artifact</artifactId>
  <version>version</version>
</dependency>

Command line tool usage