DistWalk is a simple command-line tool useful to emulate distributed processing scenarios, where a client submits requests to be processed by one or more server nodes, in a distributed environment. The client can be configured to send requests according to a pre-defined pattern (e.g., periodically or exponentially distributed), where requests may ask the server to perform processing for a time that is configurable according to a pre-defined pattern (e.g., constant or exponentially distributed processing time), or performing disk operations by reading or writing a configurable amount of data.
The client provides, at the end of the experiment, all the end-to-end processing times for all submitted requests.
The client establishes one or more TCP connections to the server, which is capable of handling multiple client connections via epoll(7). The client can submit concurrent traffic by spawning multiple threads. Furthermore, each thread can emulate different sessions where the connection is closed and re-established for each new session. The client, or the node, can forward a sequence of instructions to N different nodes and await the response of M nodes (N>=M).
This program has been developed and tested on a Linux Ubuntu system, versions from 16.xx to 21.xx. Therefore, please, use Linux.
In order to compile the client and the server, just type, from the root folder:
make
two executables are created:
The classic and simplest scenario is the one of a client-server scenario with client submitting periodically packets of a given size, and expecting back responses of a different fixed size.
This example scenario is achieved launching on the server the simple command:
[myuser@myserver distwalk/src]$ ./dw_node
then launching on the client the following command, with which we are submitting 10 packets at a rate of 1000 pkt/s
[myuser@myclient distwalk/src]$ ./src/dw_client -n 10 -r 1000 -C 1000 -ps 512
Configuration: clienthost=0.0.0.0:0 serverhost=127.0.0.1:7891 num_threads: 1 num_pkts=10 (COMPUTE:1, STORE:0, LOAD:0) rate=1000, exp_arrivals=0 period=1000us waitspin=0 ramp_num_steps=0, ramp_delta_rate=0, ramp_step_secs=0 pkt_size=512 (+66 for headers) min packet size due to header: send=20, reply=12 max packet size: 16777216 no_delay: 1 num_sessions: 1 per_session_output: 0 request template: ccmd COMPUTE(1000us)->REPLY(512b,1) t: 271 us, elapsed: 1100 us, req_id: 0, thr_id: 0, sess_id: 0 t: 1355 us, elapsed: 1026 us, req_id: 1, thr_id: 0, sess_id: 0 t: 2366 us, elapsed: 1024 us, req_id: 2, thr_id: 0, sess_id: 0 t: 3354 us, elapsed: 1045 us, req_id: 3, thr_id: 0, sess_id: 0 t: 4355 us, elapsed: 1050 us, req_id: 4, thr_id: 0, sess_id: 0 t: 5358 us, elapsed: 1065 us, req_id: 5, thr_id: 0, sess_id: 0 t: 6356 us, elapsed: 1073 us, req_id: 6, thr_id: 0, sess_id: 0 t: 7358 us, elapsed: 1081 us, req_id: 7, thr_id: 0, sess_id: 0 t: 8347 us, elapsed: 1097 us, req_id: 8, thr_id: 0, sess_id: 0 t: 9325 us, elapsed: 1136 us, req_id: 9, thr_id: 0, sess_id: 0
The following command spawns 4 threads submitting concurrently requests to the same server (each thread submits 5000 requests needing a 1ms of processing time each, at a rate of 250 pkt/s):
[myuser@myclient distwalk/src]$ ./dw_client -nt 4 -n 5000 -r 250 -C 1000
The following command spawns 3 threads, with each of them using 10 sessions to submit the 5000 packets as in the above command, so each session will send 500 requests at a rate of 250 pkt/s (at the end of each session, each client thread closes the connection and establishes a new connection for the subsequent session)
[myuser@myclient distwalk/src]$ ./dw_client -nt 3 -ns 10 -n 5000 -r 250 -C 1000