Crash with asynchronous interface

[kartiksura]

My server gets requests from the clients. For each request, I want to call 10 remote HTTP servers and process their responses.

So its going to be pretty large scale.

How would you recommend to use curl asio in such scenario? Any design hints for scaling up curl asio usage?

[kartiksura]

Also I tried to modify the code a little by trying to call it through a thread. It crashes even with number of threads = 1

#include <curl-asio.h>
#include <boost/lexical_cast.hpp>
#include <boost/thread.hpp>
#include <boost/bind.hpp>
#include <boost/make_shared.hpp>
#include <boost/ptr_container/ptr_set.hpp>
#include <iostream>
#include <fstream>
#include <memory> // for std::auto_ptr

void handle_download_completed(const boost::system::error_code& err, std::string url, curl::easy* easy)
{
    if (!err)
    {
        std::cout << "Download of " << url << " completed: Total time: %ld" << easy->get_total_time() << std::endl;
    }
    else
    {
        std::cout << "Download of " << url << " failed: " << err.message() << std::endl;
    }
}

void start_download(curl::multi& multi, const std::string& url)
{
    std::auto_ptr<curl::easy> easy(new curl::easy(multi));
    easy->set_url(url);
    easy->set_sink(boost::make_shared<std::stringstream>());
    easy->async_perform(boost::bind(handle_download_completed, boost::asio::placeholders::error, url, easy.get()));
}

int firedsp(int count, std::string url)
{
    boost::asio::io_service io_service;

    // construct an instance of curl::multi
    curl::multi manager(io_service);

    // this example program downloads all urls in the text file argv[1] to the current directory
    // start by creating an io_service object
    // treat each line in url_file_name as url and start a download from it
    while (count > 0)
    {
        count--;
        start_download(manager, url);
    }

    // let Boost.Asio do its magic
    io_service.run();

    std::cout << "Thread completed" << std::endl;

}

int main(int argc, char* argv[])
{
    if (argc != 4)
    {
        std::cerr << "Usage: loadcurl url threads calls \n";
        return 1;
    }

    std::string url(argv[1]);
    int num_threads = boost::lexical_cast<int>(argv[2]);
    int num_calls = boost::lexical_cast<int>(argv[3]);

    boost::thread_group tgroup;
    for (int i = 0; i < num_threads; i++)
    {
        tgroup.create_thread(boost::bind(firedsp, num_calls, url));
    }

    tgroup.join_all();
    return 0;
}

(gdb) bt
#0  0x00000000004116eb in boost::asio::detail::task_io_service_operation* boost::asio::detail::op_queue_access::next<boost::asio::detail::task_io_service_operation>(boost::asio::detail::task_io_service_operation*) ()
#1  0x000000000040f60e in boost::asio::detail::op_queue<boost::asio::detail::task_io_service_operation>::pop() ()
#2  0x000000000040c811 in boost::asio::detail::task_io_service::shutdown_service() ()
#3  0x000000000040bd20 in boost::asio::detail::service_registry::~service_registry() ()
#4  0x000000000040cfd5 in boost::asio::io_service::~io_service() ()
#5  0x00007f1b39aef219 in __run_exit_handlers () from /lib64/libc.so.6
#6  0x00007f1b39aef265 in exit () from /lib64/libc.so.6
#7  0x00007f1b39ad7afc in __libc_start_main () from /lib64/libc.so.6
#8  0x000000000040a069 in _start ()

[mologie]

Great, I can reproduce this issue. I have edited the report and your second comment slightly for formatting and a proper description - I hope you don't mind. Please use the preview button next time and read the formatting guide.

I will look into this issue in the next few days. Cheers and thanks for the report.