search

tempesta-tech / tempesta

All-in-one solution for high performance web content delivery and advanced protection against DDoS and web attacks
https://tempesta-tech.com/
GNU General Public License v2.0
618 stars 103 forks source link

tests: tests for access log transmission to Clickhouse server #2269

Open ai-tmpst opened 1 week ago

ai-tmpst commented 1 week ago

To verify the correct operation of the access log transmission from the Tempesta FW ring buffer, mapped to userspace, to a Clickhouse server via a user-space utility utils/tfw_logger. This test will ensure that access logs are correctly generated by Tempesta FW, and transmitted to the Clickhouse server without data loss or corruption.

Clickhouse has SQL-like query syntax.

Access log table creation: CREATE TABLE IF NOT EXISTS access_log (timestamp UInt64, address IPv6, method UInt8, version UInt8, status UInt16, response_content_length UInt32, response_time UInt32, vhost String, uri String, referer String, user_agent String, PRIMARY KEY(timestamp))

For tests I used Clickhouse client/server downloaded as curl https://clickhouse.com/ | sh. But I think it would be better to install the server from distro repositories.

krizhanovsky commented 1 week ago

I'll deploy the Clockhouse logs on our website, so there will be manual testing and we can move the task to 0.9