search

tempesta-tech / tempesta-test

Test suite for Tempesta FW
10 stars 4 forks source link

Add CurlClient: wrapper to manage cURL #332

Closed b3b closed 1 year ago

b3b commented 1 year ago

Add new framework.client.Client for convenient work with the curl utility. Client was used in #290, and now pulled into separate PR.

Client implementation includes:

Client declaration examples

 clients = [
        # GET http://{tempesta.ip}
        {
            "id": "default",
            "type": "curl",
        },

        # HEAD https://{tempesta.ip}:8000/page , with verbose output
        {
            "id": "ssl-head",
            "type": "curl",
            "ssl": True,
            "addr": "${tempesta_ip}:8000",
            "uri": "/page"
            # additional cURL binary arguments
            "cmd_args": (
                " --head"
                " --verbose"
            ),
        },

        # HTTP/2 POST data with cookies enabled
        {
            "id": "h2-post",
            "type": "curl",
            "http2": True,
            "data": "param1=1&param2=2",
            "save_cookies": True,
            "load_cookies": True,
        },

        # Make requests to /1, /2 ... /100 , using 10 simultaneous transfers,
        # with disabled output and "Connection: close" header set
        {
            "id": "parallel",
            "type": "curl",
            "uri": "/[1-100]",
            "parallel": 10,
            "disable_output": True,
            "headers": {
                "Connection": "close",
            },
        },
]

Log output example

Output contains full cURL command line, for easy manual reproduction, and information about performed transfers.

Running client Curl command formatted: curl --no-progress-meter --create-dirs --dump-header '/tmp/client/curl-default.hdr' --write-out '%{json}' --output '/tmp/client/curl-output' --cookie-jar '/tmp/client/curl-default.jar' --cookie '/tmp/client/curl-default.jar' --http1.1 'http://192.168.122.76/' Client exit Stopping client client stdout: {"content_type":null,"errormsg":null,"exitcode":0,"filename_effective":"/tmp/client/curl-output","ftp_entry_path":null,"http_code":200,"http_connect":0,"http_version":"1.1","local_ip":"192.168.122.1","local_port":52846,"method":"GET","num_connects":1,"num_headers":7,"num_redirects":0,"proxy_ssl_verify_result":0,"redirect_url":null,"referer":null,"remote_ip":"192.168.122.76","remote_port":80,"response_code":200,"scheme":"HTTP","size_download":4,"size_header":198,"size_request":78,"size_upload":0,"speed_download":492,"speed_upload":0,"ssl_verify_result":0,"time_appconnect":0.000000,"time_connect":0.000249,"time_namelookup":0.000017,"time_pretransfer":0.000279,"time_redirect":0.000000,"time_starttransfer":0.008015,"time_total":0.008130,"url":"http://192.168.122.76/","url_effective":"http://192.168.122.76/","urlnum":0,"curl_version":"libcurl/7.85.0 OpenSSL/3.0.2 zlib/1.2.11 nghttp2/1.43.0"} Client is stopped

Usage example

From the tester.TempestaTest method context:

>>> client = self.get_client("curl")
>>> client.start()
>>> self.wait_while_busy(client)
>>> client.stop()

# Client object representation
>>> client
CurlClient(addr='192.168.122.76', uri='http://192.168.122.76/', cmd_args='', data='',
    headers=None, dump_headers=True, disable_output=False, save_cookies=True,
    load_cookies=True, ssl=False, http2=False, insecure=True, parallel=None)

# Observe the responses list
>>> client.responses
[CurlResponse(status=200, proto='1.1',
    headers={'date': 'test', 'x-header': 'Test-Value-2', 'set-cookie': 'curl=test', 'content-length': '4',
    'via': '1.1 tempesta_fw (Tempesta FW pre-0.7.0)', 'server': 'Tempesta FW/pre-0.7.0'})]

# Observe the last response
>>> client.last_response
CurlResponse(status=200, proto='1.1',
    headers={'date': 'test', 'x-header': 'Test-Value-2', 'set-cookie': 'curl=test', 'content-length': '4',
    'via': '1.1 tempesta_fw (Tempesta FW pre-0.7.0)', 'server': 'Tempesta FW/pre-0.7.0'})
>>> client.last_response.status
200

# Observe received headers
>>> client.last_response.headers
{'date': 'test', 'x-header': 'Test-Value-2', 'set-cookie': 'curl=test', 'content-length': '4',
    'via': '1.1 tempesta_fw (Tempesta FW pre-0.7.0)', 'server': 'Tempesta FW/pre-0.7.0'}

# Observe headers with multiple values, like cookies
>>> client.last_response.multi_headers
{'date': ['test'], 'x-header': ['Test-Value-1', 'Test-Value-2'],
    'set-cookie': ['curl=test'], 'content-length': ['4'], 'via': ['1.1 tempesta_fw (Tempesta FW pre-0.7.0)'],
    'server': ['Tempesta FW/pre-0.7.0']}

# Discover transfers stats
>>> len(client.stats)
1
>>> client.last_stats
{'content_type': None, 'errormsg': None, 'exitcode': 0, 'filename_effective': '/tmp/client/curl-output',
    'ftp_entry_path': None, 'http_code': 200, 'http_connect': 0, 'http_version': '1.1',
    'local_ip': '192.168.122.1', 'local_port': 46732, 'method': 'GET', 'num_connects': 1,
    'num_headers': 7, 'num_redirects': 0, 'proxy_ssl_verify_result': 0, 'redirect_url': None,
    'referer': None, 'remote_ip': '192.168.122.76', 'remote_port': 80, 'response_code': 200,
    'scheme': 'HTTP', 'size_download': 4, 'size_header': 198, 'size_request': 97, 'size_upload': 0,
    'speed_download': 112, 'speed_upload': 0, 'ssl_verify_result': 0, 'time_appconnect': 0.0,
    'time_connect': 0.000245, 'time_namelookup': 1.2e-05, 'time_pretransfer': 0.000266,
    'time_redirect': 0.0, 'time_starttransfer': 0.035547, 'time_total': 0.035635,
    'url': 'http://192.168.122.76/', 'url_effective': 'http://192.168.122.76/', 'urlnum': 0,
    'curl_version': 'libcurl/7.85.0 OpenSSL/3.0.2 zlib/1.2.11 nghttp2/1.43.0'
}

# Change uri for the next request
>>> client.set_uri("/page2")
# Add additional option (set request method to PURGE)
>>> client.options.append("--request PURGE")
>>> client.start()

Version check

сURL binary expected version is set by the framework.curl_client.CURL_BINARY_VERSION. On version mismatch, every request will produce error messages:

helpers.error.Error: Expected curl binary version: 7.85.0
Detected curl binary version: 7.68.0
Set 'Client.curl_version' config variable to override expected value.

or, for badly outdated versions:

helpers.error.Error: Can't detect `curl` version. `curl --version` should be 7.85.0
b3b commented 1 year ago

Installed curl version check added. Installation from sources added to the setup.sh.