Load Testing Framework for Distributed Applications built on Top of Yarn
DemandSpike is designed as a command of the cluster shell. The demandspike command has several subcommand and it can be used with the existing cluster command to create a powerful test script.
Send Command
demandspike:job send --max-num-of-message 1000
demandspike:job send
--driver kafka --broker-connect 127.0.0.1:9092
--topic metrics.consumer --max-num-of-message 1000
service failure simulation command
demandspike:job simulation
--name service-failure --target-member-role kafka
--module Kafka --service-id KafkaClusterService --delay 0 --period 5000 --failure-time 1000
TODO: Implement a send command that start an app master on yarn to create a load test for kafka and sparkngin
DemandSpike job is a test script that can be queued and run by the job scheduler. A demandspike job has the main attributes such job id, job description , the script to run. When a job is finished , the job runner can store some information such the output, the cluster metrics... with the demandspike job.
To install the DemandSpike Job Service
module install --member-role demandspike --autostart --module DemandSpike
To submit a job to the job service
demandspike submit --member-name demandspike --file demandspikejob.json
where the demandspikejob.json
{
"id": "1",
"description": "Sample DemandSpike job",
"tasks": [
{
"description": "clean metric task",
"command": "server metric-clear --expression *"
},
{
"description": "send by the dummy driver",
"command": "demandspike:job send --max-num-of-message 1000"
},
{
"description": "Run service failure simulation",
"command": "demandspike:job simulation --name service-failure --target-member-role kafka --module Kafka --service-id KafkaClusterService --delay 3000"
},
{
"description": "send by the kafka driver",
"command": "demandspike:job send --driver kafka --broker-connect 127.0.0.1:9092 --topic metrics.consumer --max-num-of-message 1000"
}
]
}
To get the job service scheduler info
demandspike scheduler --member-name demandspike
To uninstall the demandspike job service
module uninstall --member-role demandspike --timeout 20000 --module DemandSpike
To submit a demandspike job to the job service
To generate the eclipse configuration
To import the project into the eclipse
Build the release
gradle clean build install release
Start demandSpike job
cd build/release/DemandSpike/bin
#To launch the server
./server.sh
#Ping to check the server status
./shell.sh -c server ping
#To launch the batch script tets
./shell.sh -f hello-demandspike.csh