Load testing voice systems, and voice applications in particular, is tricky. While several commercial tools exist, there is really only one tool in the Open Source world that is good at efficiently generating SIP load: SIPp. While SIPp does a good job of generating load, it is somewhat clumsy to use, due to a verbose XML format for scenarios, a confusing set of command line parameters, and worst of all, a lack of tools to create media needed to interact with voice applications.
The last problem is especially tricky: Imagine you want to load test an IVR. Testing requires:
To test this with SIPp you need a PCAP file that contains the properly timed DTMF interactions. Since there is no tool to create this media, it is usually necessary to call into the system and record the PCAP, isolate the RTP from the captured packets with something like Wireshark, then connect the pcap file into the SIPp scenario. This process is time consuming and error prone, meaning that testing isn't done as often as it should.
SippyCup aims to help solve these problems.
Sippy Cup is a tool to generate SIPp load test profiles and the corresponding media in PCAP format. The goal is to take an input document that describes a load test in a very simple way (call this number, wait this many seconds, send this digit, wait a few more seconds, etc). The ideas are taken from LoadBot, but the goal is for a more performant load generating tool with no dependency on Asterisk.
SippyCup relies on the following to generate scenarios and the associated media PCAP files:
If you do not have Ruby 2.3.3 available (check using ruby --version
), we recommend installing Ruby with RVM
Once Ruby is installed, install SippyCup:
gem install sippy_cup
Now you can start creating scenario files like in the examples below.
You use bundle
command (from the "bundler" package) to install from the source directly. First, clone the repository into a working directory.
Install bundle
via gem:
gem install bundler --no-ri --no-rdoc
Then build the sippy_cup
application with bundle
.
bundle install
Using bundle
will then install the gem dependencies and allow you to run sippy_cup
from your working directory.
---
source: 192.0.2.15
destination: 192.0.2.200
max_concurrent: 10
calls_per_second: 5
number_of_calls: 20
steps:
- invite
- wait_for_answer
- ack_answer
- sleep 3
- send_digits '3125551234'
- sleep 5
- send_digits '#'
- wait_for_hangup
Both source
and destination
above may be optionally supplied with a port number, eg. 192.0.2.200:5061
Next, execute the scenario:
$ sippy_cup -r my_test_scenario.yml
I, [2013-09-30T14:48:08.388106 #9883] INFO -- : Preparing to run SIPp command: sudo sipp -i 192.0.2.15 -p 8836 -sf /var/folders/n4/dpzsp6_95tb3c4sp12xj5wdr0000gn/T/scenario20130930-9883-1crejcw -l 10 -m 20 -r 5 -s 1 192.0.2.200
Password:
...snip...
I, [2013-09-30T14:48:16.728712 #9883] INFO -- : Test completed successfully.
More examples are available in the source repository.
require 'sippy_cup'
scenario = SippyCup::Scenario.new 'Sippy Cup', source: '192.168.5.5:10001', destination: '10.10.0.3:19995' do |s|
s.invite
s.wait_for_answer
s.ack_answer
s.sleep 3
s.send_digits '3125551234'
s.sleep 5
s.send_digits '#'
s.wait_for_hangup
end
# Create the scenario XML and PCAP media. File will be named after the scenario name, in our case:
# * sippy_cup.xml
# * sippy_cup.pcap
scenario.compile!
The above code can be executed as a standalone Ruby script and the resulting scenario file run with SIPp.
Each command below can take SIPp attributes as optional arguments. For a full list of available steps with arguments explained, see the API documentation.
sleep <seconds>
Wait a specified number of secondsinvite
Send a SIP INVITE to the specified targetreceive_invite
Wait for an INVITE to be receivedregister <username> [password]
Register the specified user to the target with an optional passwordsend_trying
Send a 100 Trying
provisional responsereceive_trying
Expect to receive a 100 Trying
response from the targetsend_ringing
Send a 180 Ringing
provisional responsereceive_ringing
Expect to receive a 180 Ringing
response from the targetreceive_progress
Expect to receive a 183 Progress
response from the targetsend_answer
Send a 200 Ok
response to an INVITE (answer the call)receive_answer
Expect to receive a 200 OK
(answering the call) response from the targetanswer
Convenient shortcut for send_answer; receive_ack
wait_for_answer
Convenient shortcut for receive_trying; receive_ringing; receive_progress; receive_answer
, with all but the answer
marked as optionalack_answer
Send an ACK
in response to a 200 OK
receive_ack
Expect to receive an ACK
send_digits <string>
Send a DTMF string. May send one or many digits, including 0-9
, *
, #
, and A-D
receive_ok
Expect to receive a 200 OK
receive_message [regex]
Expect to receive a SIP MESSAGE, optionally matching a regexsend_bye
Send a BYE
(hangup request)receive_bye
Expect to receive a BYE
from the targetack_bye
Send a 200 OK
response to a BYE
wait_for_hangup
Convenient shortcut for receive_bye; ack_bye
hangup
Convenient shortcut for send_bye; receive_ok
call_length_repartition
Creates a histogram table of individual call lengths in milliseconds between min length and max length, at the specified intervalresponse_time_repartition
Creates a histogram table of individual SIP request response times in milliseconds between min length and max length, at the specified intervalDon't want your scenario to end up in the same directory as your script? Need the filename to be different than the scenario name? No problem!
For the sippy_cup
manifest, use filename
:
---
filename: /path/to/somewhere
Or, in Ruby:
s = SippyCup::Scenario.new 'SippyCup', source: '192.168.5.5:10001', destination: '10.10.0.3:19995', filename: '/path/to/somewhere' do
# scenario definitions here...
end
s.compile!
This will create the files somewhere.xml
and somewhere.pcap
in the /path/to/
directory.
Each parameter has an impact on the test, and may either be changed once the XML file is generated or specified in the options hash for SippyCup::Scenario.new
. In addition to the default parameters, some additional parameters can be set:
With Sippy Cup, you can add additional attributes to each step of the scenario:
# This limits the amount of time the server has to reply to an invite (3 seconds)
s.receive_answer timeout: 3000
# You can override the default 'optional' parameters
s.receive_ringing optional: false
s.receive_answer optional: true
# Let's combine multiple attributes...
s.receive_answer timeout: 3000, crlf: true
For more information on possible attributes, visit the SIPp Documentation.
Copyright (C) 2013-2015 Mojo Lingo LLC
Sippy Cup is released under the MIT license. Please see the LICENSE file for details.
Sippy Cup was created by Ben Klang and Will Drexler with support from Mojo Lingo and their clients.
"Sippy Cup" name suggested by Jamey Owens