tests: tests for access log transmission to Clickhouse server

[ai-tmpst]

To verify the correct operation of the access log transmission from the Tempesta FW ring buffer, mapped to userspace, to a Clickhouse server via a user-space utility utils/tfw_logger. This test will ensure that access logs are correctly generated by Tempesta FW, and transmitted to the Clickhouse server without data loss or corruption.

Clickhouse has SQL-like query syntax.

Access log table creation: CREATE TABLE IF NOT EXISTS access_log (timestamp DateTime64, address IPv6, method UInt8, version UInt8, status UInt16, response_content_length UInt32, response_time UInt32, vhost String, uri String, referer String, user_agent String, dropped_events UInt64, PRIMARY KEY(timestamp))

For tests I used Clickhouse client/server downloaded as curl https://clickhouse.com/ | sh. But I think it would be better to install the server from distro repositories.

[krizhanovsky]

I'll deploy the Clockhouse logs on our website, so there will be manual testing and we can move the task to 0.9.

Things to test:

• [ ] access log with dmesg and clickouse or clickhouse only
• [ ] test different values for mmap_log_buffer_size
• [ ] restart Tempesta FW under load (e.g. h2load or wrk) with clickhouse access log
• [ ] query Clickhouse that all the records are presented and correct (URL, IP addresses, Host, response status and size, timestamps are monotonically increasing and do represent correct current time)
• [ ] send SIGSTOP for the user space daemon, wait a little under load, and send SIGCONT - check that Clickhouse entries represent the number of records, which couldn't be logged
• [ ] with deproxy introduce a small delay on the server side and check that response time correctly represents the delay in this case and no-delay cases

[ai-tmpst]

for i in $(seq 1 30); do kill -SIGSTOP 8384 && sleep 10 && kill -SIGCONT 8384; done Something like this we need in tests. Now it can lead to incorrect tail position.