chriskohlhoff / asio

Asio C++ Library
http://think-async.com/Asio
4.97k stars 1.22k forks source link

Possible degradation in asio performance since Boost version 1.56 (Centos 6.x, x64) #635

Open ghost opened 3 years ago

ghost commented 3 years ago

@plynch1976 commented on Sep 4, 2018, 2:59 PM UTC:

Hi, We’ve been using Boost for the last number of years (specifically Boost.asio) and for the most part it’s been fantastic for us. We’ve just upgraded an older service to use a later Boost version (“service A” was written 5 years ago and it was based on boost.asio using ssl).

We noticed a significant drop in Queries Per Second when we performance tested the service with the newer Boost, so we investigated further. See below for details.

Is there something simple that we are missing – e.g. the compiler flags with the newer Boost? Has this issue been seen before?

“Service A” code is closely based on the example shown here https://www.boost.org/doc/libs/1_39_0/doc/html/boost_asio/example/ssl/server.cpp

Any help or pointers here would be greatly appreciated. For the moment, I have had to drop the services back to use Boost 1.55

Thanks, Paul.

Summary of the findings: Boost ASIO performance has degraded significantly for us since Boost version 1.55, with further performance drops until 1.63. I have not tried the latest boost version 1.68

We have a service that was built 5 years ago & it used Boost 1.51. In a cleanup operation, we upgraded Boost to version 1.63 and noticed significant performance drops against what we had previously (throughput on the server dropped from ~50k per second to ~16k per second with no other code changes).

Further investigation using iterative upgrades from 1.51, to 1.52 all the way to 1.63 showed that the first performance impact was in 1.56 where performance dropped from ~50k QPS to 31k QPS. By 1.63, the performance had dropped to ~16k as mentioned above.

We have noticed this same drop across two different services (both deployed & built on Centos 6.x, x64):

  1. “Service A” (mentioned above) is a binary service over SSL using Boost.Asio. We use custom load test tools to test the maximum threshold of the service. Each request payload is < 1kb.
  1. “Service B” is a service based on RESTful service based on nginx & Boost.Asio. We use Tsung to test the maximum threshold of this service. Nginx does the SSL termination in this service and forwards to the FCGI library written using boost asio. Each request payload is < 1kb.

There is no shared code in either of the services apart from the fact that they are both based on ASIO TCP servers

**Boost version Performance (QPS)**
1.63 performance Total:  15,749
1.57 performance Total:  30,923
1.51 performance Total:  49,421

Compiler We have tried with both gcc/g++ 4.4.7 and dev toolset 3. The same performance impacts are seen

dev toolset3 $ gcc --version gcc (GCC) 4.9.2 20150212 (Red Hat 4.9.2-6) Copyright (C) 2014 Free Software Foundation, Inc.

Build We build Boost from source for every build and link dynamically. Note that to build some of the later boost versions I removed the "cat" command.

cat < ./tools/build/v2/user-config.jam using gcc : : ${_GCCHOME}/g++; EOF ./bootstrap.sh --prefix=$temp_install_dir ./b2 install --prefix=$temp_install_dir --toolset=gcc link=static

Server config

This issue was moved by chriskohlhoff from boostorg/asio#141.

ljluestc commented 3 weeks ago

#include <asio.hpp>
#include <asio/ssl.hpp>
#include <iostream>
#include <memory>

class ssl_server {
public:
    ssl_server(asio::io_context& io_context, asio::ssl::context& ssl_context)
        : acceptor_(io_context, asio::ip::tcp::endpoint(asio::ip::tcp::v4(), 12345)),
          ssl_context_(ssl_context) {
        start_accept();
    }

private:
    void start_accept() {
        auto socket = std::make_shared<asio::ssl::stream<asio::ip::tcp::socket>>(acceptor_.get_executor().context(), ssl_context_);
        acceptor_.async_accept(socket->lowest_layer(), 
            [this, socket](const asio::error_code& error) {
                if (!error) {
                    start_handshake(socket);
                }
                start_accept(); // Accept the next connection
            });
    }

    void start_handshake(std::shared_ptr<asio::ssl::stream<asio::ip::tcp::socket>> socket) {
        socket->async_handshake(asio::ssl::stream_base::server,
            [this, socket](const asio::error_code& error) {
                if (!error) {
                    start_read(socket);
                } else {
                    std::cerr << "Handshake error: " << error.message() << std::endl;
                }
            });
    }

    void start_read(std::shared_ptr<asio::ssl::stream<asio::ip::tcp::socket>> socket) {
        auto buffer = std::make_shared<std::array<char, 1024>>();
        socket->async_read_some(asio::buffer(*buffer),
            [this, socket, buffer](const asio::error_code& error, std::size_t bytes_transferred) {
                if (!error) {
                    // Process the data...
                    start_write(socket, buffer, bytes_transferred);
                } else {
                    std::cerr << "Read error: " << error.message() << std::endl;
                }
            });
    }

    void start_write(std::shared_ptr<asio::ssl::stream<asio::ip::tcp::socket>> socket, 
                     std::shared_ptr<std::array<char, 1024>> buffer, 
                     std::size_t bytes_transferred) {
        asio::async_write(*socket, asio::buffer(*buffer, bytes_transferred),
            [this, socket](const asio::error_code& error, std::size_t) {
                if (error) {
                    std::cerr << "Write error: " << error.message() << std::endl;
                }
                start_read(socket); // Read next data
            });
    }

    asio::ip::tcp::acceptor acceptor_;
    asio::ssl::context& ssl_context_;
};

int main() {
    try {
        asio::io_context io_context;
        asio::ssl::context ssl_context(asio::ssl::context::sslv23);

        // Load your SSL certificates and keys
        ssl_context.use_certificate_chain_file("server.crt");
        ssl_context.use_private_key_file("server.key", asio::ssl::context::pem);
        ssl_context.use_tmp_dh_file("dh2048.pem");

        ssl_server server(io_context, ssl_context);
        io_context.run();
    } catch (const std::exception& e) {
        std::cerr << "Error: " << e.what() << std::endl;
    }

    return 0;
}