basiliscos / cpp-bredis

Boost::ASIO low-level redis client (connector)
MIT License
147 stars 36 forks source link

Wrapping sockets with Bredis::Connection #43

Closed BarbBoyaji closed 4 years ago

BarbBoyaji commented 4 years ago

Hi, I was initially using boost version 1.73. Wrapping the underlying type boost::asio::ip::tcp::socket as a bredis connection works. However, doing the same for the unix sockets with type boost::asio::local::stream_protocol::socket does not.

This is the code I am running at the moment, which fails during building.

boost::asio::io_context ioserv;
using socket_t = boost::asio::local::stream_protocol::socket;
using next_layer_t = socket_t;

boost::asio::local::stream_protocol::endpoint end_point("/shared/redis.sock");
socket_t sock(ioserv); 
sock.connect(end_point);
bredis::Connection<next_layer_t> bredis_con(std::move(socket));

And here is the error I get.

In file included from /usr/local/include/boost/asio/executor.hpp:338,
                 from /usr/local/include/boost/asio/basic_socket.hpp:27,
                 from /usr/local/include/boost/asio/basic_datagram_socket.hpp:20,
                 from /usr/local/include/boost/asio.hpp:24,
                 from /atom/languages/cpp/third-party/cpp-bredis/include/bredis/Connection.hpp:20,
                 from ./clients/bredis_client.cc:7:
/usr/local/include/boost/asio/impl/executor.hpp: In instantiation of 'boost::asio::execution_context& boost::asio::executor::impl< <template-parameter-1-1>, <template-parameter-1-2> >::context() [with Executor = int (*)(int, int, int); Allocator = std::allocator<void>]':
/usr/local/include/boost/asio/impl/executor.hpp:177:22:   required from here
/usr/local/include/boost/asio/impl/executor.hpp:179:22: error: request for member 'context' in '((boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >*)this)->boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >::executor_', which is of non-class type 'int (*)(int, int, int)'
     return executor_.context();
            ~~~~~~~~~~^~~~~~~
/usr/local/include/boost/asio/impl/executor.hpp: In instantiation of 'void boost::asio::executor::impl< <template-parameter-1-1>, <template-parameter-1-2> >::on_work_started() [with Executor = int (*)(int, int, int); Allocator = std::allocator<void>]':
/usr/local/include/boost/asio/impl/executor.hpp:167:8:   required from here
/usr/local/include/boost/asio/impl/executor.hpp:169:15: error: request for member 'on_work_started' in '((boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >*)this)->boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >::executor_', which is of non-class type 'int (*)(int, int, int)'
     executor_.on_work_started();
     ~~~~~~~~~~^~~~~~~~~~~~~~~
/usr/local/include/boost/asio/impl/executor.hpp: In instantiation of 'void boost::asio::executor::impl< <template-parameter-1-1>, <template-parameter-1-2> >::on_work_finished() [with Executor = int (*)(int, int, int); Allocator = std::allocator<void>]':
/usr/local/include/boost/asio/impl/executor.hpp:172:8:   required from here
/usr/local/include/boost/asio/impl/executor.hpp:174:15: error: request for member 'on_work_finished' in '((boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >*)this)->boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >::executor_', which is of non-class type 'int (*)(int, int, int)'
     executor_.on_work_finished();
     ~~~~~~~~~~^~~~~~~~~~~~~~~~
/usr/local/include/boost/asio/impl/executor.hpp: In instantiation of 'void boost::asio::executor::impl< <template-parameter-1-1>, <template-parameter-1-2> >::dispatch(boost::asio::executor::function&&) [with Executor = int (*)(int, int, int); Allocator = std::allocator<void>]':
/usr/local/include/boost/asio/impl/executor.hpp:182:8:   required from here
/usr/local/include/boost/asio/impl/executor.hpp:184:15: error: request for member 'dispatch' in '((boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >*)this)->boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >::executor_', which is of non-class type 'int (*)(int, int, int)'
     executor_.dispatch(BOOST_ASIO_MOVE_CAST(function)(f), allocator_);
     ~~~~~~~~~~^~~~~~~~
/usr/local/include/boost/asio/impl/executor.hpp: In instantiation of 'void boost::asio::executor::impl< <template-parameter-1-1>, <template-parameter-1-2> >::post(boost::asio::executor::function&&) [with Executor = int (*)(int, int, int); Allocator = std::allocator<void>]':
/usr/local/include/boost/asio/impl/executor.hpp:187:8:   required from here
/usr/local/include/boost/asio/impl/executor.hpp:189:15: error: request for member 'post' in '((boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >*)this)->boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >::executor_', which is of non-class type 'int (*)(int, int, int)'
     executor_.post(BOOST_ASIO_MOVE_CAST(function)(f), allocator_);
     ~~~~~~~~~~^~~~
/usr/local/include/boost/asio/impl/executor.hpp: In instantiation of 'void boost::asio::executor::impl< <template-parameter-1-1>, <template-parameter-1-2> >::defer(boost::asio::executor::function&&) [with Executor = int (*)(int, int, int); Allocator = std::allocator<void>]':
/usr/local/include/boost/asio/impl/executor.hpp:192:8:   required from here
/usr/local/include/boost/asio/impl/executor.hpp:194:15: error: request for member 'defer' in '((boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >*)this)->boost::asio::executor::impl<int (*)(int, int, int), std::allocator<void> >::executor_', which is of non-class type 'int (*)(int, int, int)'
     executor_.defer(BOOST_ASIO_MOVE_CAST(function)(f), allocator_);
     ~~~~~~~~~~^~~~~

I tried downgrading to boost 1.69, but got the following error:

/atom/languages/cpp/third-party/cpp-bredis/include/bredis/Connection.hpp: In instantiation of 'bredis::Connection<NextLayer>::Connection(Args&& ...) [with Args = {int (&)(int, int, int)}; NextLayer = boost::asio::basic_stream_socket<boost::asio::local::stream_protocol>]':
./clients/bredis_client.cc:61:66:   required from here
/atom/languages/cpp/third-party/cpp-bredis/include/bredis/Connection.hpp:41:46: error: no matching function for call to 'boost::asio::basic_stream_socket<boost::asio::local::stream_protocol>::basic_stream_socket(int (&)(int, int, int))'
         : stream_(std::forward<Args>(args)...) {}
                                              ^
In file included from /usr/local/include/boost/asio/basic_socket_streambuf.hpp:25,
                 from /usr/local/include/boost/asio/basic_socket_iostream.hpp:24,
                 from /usr/local/include/boost/asio.hpp:31,
                 from /atom/languages/cpp/third-party/cpp-bredis/include/bredis/Connection.hpp:20,
                 from ./clients/bredis_client.cc:7:
/usr/local/include/boost/asio/basic_stream_socket.hpp:185:3: note: candidate: 'template<class Protocol1> boost::asio::basic_stream_socket<Protocol>::basic_stream_socket(boost::asio::basic_stream_socket<Protocol1>&&, typename std::enable_if<std::is_convertible<Protocol1, Protocol>::value>::type*)'
   basic_stream_socket(
   ^~~~~~~~~~~~~~~~~~~
/usr/local/include/boost/asio/basic_stream_socket.hpp:185:3: note:   template argument deduction/substitution failed:
In file included from ./clients/bredis_client.cc:7:
/atom/languages/cpp/third-party/cpp-bredis/include/bredis/Connection.hpp:41:46: note:   mismatched types 'boost::asio::basic_stream_socket<Protocol>' and 'int(int, int, int)'
         : stream_(std::forward<Args>(args)...) {}
                                              ^
In file included from /usr/local/include/boost/asio/basic_socket_streambuf.hpp:25,
                 from /usr/local/include/boost/asio/basic_socket_iostream.hpp:24,
                 from /usr/local/include/boost/asio.hpp:31,
                 from /atom/languages/cpp/third-party/cpp-bredis/include/bredis/Connection.hpp:20,
                 from ./clients/bredis_client.cc:7:
/usr/local/include/boost/asio/basic_stream_socket.hpp:152:3: note: candidate: 'boost::asio::basic_stream_socket<Protocol>::basic_stream_socket(boost::asio::basic_stream_socket<Protocol>&&) [with Protocol = boost::asio::local::stream_protocol]'
   basic_stream_socket(basic_stream_socket&& other)
   ^~~~~~~~~~~~~~~~~~~
/usr/local/include/boost/asio/basic_stream_socket.hpp:152:3: note:   no known conversion for argument 1 from 'int(int, int, int)' to 'boost::asio::basic_stream_socket<boost::asio::local::stream_protocol>&&'
/usr/local/include/boost/asio/basic_stream_socket.hpp:134:3: note: candidate: 'boost::asio::basic_stream_socket<Protocol>::basic_stream_socket(boost::asio::io_context&, const protocol_type&, const native_handle_type&) [with Protocol = boost::asio::local::stream_protocol; boost::asio::basic_stream_socket<Protocol>::protocol_type = boost::asio::local::stream_protocol; boost::asio::basic_stream_socket<Protocol>::native_handle_type = int]'
   basic_stream_socket(boost::asio::io_context& io_context,
   ^~~~~~~~~~~~~~~~~~~
/usr/local/include/boost/asio/basic_stream_socket.hpp:134:3: note:   candidate expects 3 arguments, 1 provided
/usr/local/include/boost/asio/basic_stream_socket.hpp:114:3: note: candidate: 'boost::asio::basic_stream_socket<Protocol>::basic_stream_socket(boost::asio::io_context&, const endpoint_type&) [with Protocol = boost::asio::local::stream_protocol; boost::asio::basic_stream_socket<Protocol>::endpoint_type = boost::asio::local::basic_endpoint<boost::asio::local::stream_protocol>]'
   basic_stream_socket(boost::asio::io_context& io_context,
   ^~~~~~~~~~~~~~~~~~~
/usr/local/include/boost/asio/basic_stream_socket.hpp:114:3: note:   candidate expects 2 arguments, 1 provided
/usr/local/include/boost/asio/basic_stream_socket.hpp:93:3: note: candidate: 'boost::asio::basic_stream_socket<Protocol>::basic_stream_socket(boost::asio::io_context&, const protocol_type&) [with Protocol = boost::asio::local::stream_protocol; boost::asio::basic_stream_socket<Protocol>::protocol_type = boost::asio::local::stream_protocol]'
   basic_stream_socket(boost::asio::io_context& io_context,
   ^~~~~~~~~~~~~~~~~~~
/usr/local/include/boost/asio/basic_stream_socket.hpp:93:3: note:   candidate expects 2 arguments, 1 provided
/usr/local/include/boost/asio/basic_stream_socket.hpp:76:12: note: candidate: 'boost::asio::basic_stream_socket<Protocol>::basic_stream_socket(boost::asio::io_context&) [with Protocol = boost::asio::local::stream_protocol]'
   explicit basic_stream_socket(boost::asio::io_context& io_context)
            ^~~~~~~~~~~~~~~~~~~
/usr/local/include/boost/asio/basic_stream_socket.hpp:76:12: note:   no known conversion for argument 1 from 'int(int, int, int)' to 'boost::asio::io_context&'
make: *** [Makefile:102: clients/build/bredis_client] Error 1

I'm running inside of a Docker container with Debian GNU/Linux 10 (buster).

basiliscos commented 4 years ago

Could you please, send an minimal example with main.cpp ?

BarbBoyaji commented 4 years ago

Hello, yes I would be happy to. I get the build errors with this file.

#include <iostream>
#include <chrono>
#include <string>
#include <fstream>
#include <vector>

#include <bredis/Connection.hpp>
#include <bredis/Extract.hpp>
#include <bredis/MarkerHelpers.hpp>
#include <boost/asio.hpp>
#include <boost/algorithm/string.hpp>

int main(){

    using Buffer = boost::asio::streambuf;
    using Iterator = typename bredis::to_iterator<Buffer>::iterator_t;

    boost::asio::io_context ioserv;

    // unix socket
    using socket_t = boost::asio::local::stream_protocol::socket;
    using next_layer_t = socket_t;

    boost::asio::local::stream_protocol::endpoint end_point("/shared/redis.sock");
    socket_t sock(ioserv);
    sock.connect(end_point);

    std::cout<<"Redis client is connected!"<<std::endl;

    //wrap socket in bredis connection
    bredis::Connection<next_layer_t> bredis_con(std::move(socket));

    //create buffer
    Buffer tx_buff, rx_buff;

    //read image in
    std::ifstream img_file("./clients/data/nasa1.jpg", std::ios::binary);
    std::string img_binary = std::string((std::istreambuf_iterator<char>(img_file)), std::istreambuf_iterator<char>());

    //XADD 
    bredis_con.write(bredis::single_command_t{ "XADD", "bredis", "*", "nasa_pic", img_binary });

    auto result_markers = bredis_con.read(rx_buff);

    auto extract = boost::apply_visitor(bredis::extractor<Iterator>(), result_markers.result);
    try{
        auto &reply_str = boost::get<bredis::extracts::string_t>(extract);
    }catch(boost::bad_get &err){
            std::cout<<"ERROR:" << ": "<< err.what()<< std::endl;
            auto &reply_str = boost::get<bredis::extracts::error_t>(extract);
            std::cout<<reply_str.str<<std::endl;
    }
    // consume the buffers, after finishing work with the markers
    rx_buff.consume(result_markers.consumed);

    //XRANGE
    bredis_con.write(bredis::single_command_t{ "XRANGE" , "bredis",  "-", "+", "COUNT", "10"});

    auto result_markers2 = bredis_con.read(tx_buff);

    auto extract2 = boost::apply_visitor(bredis::extractor<Iterator>(), result_markers.result);
    tx_buff.consume(result_markers2.consumed);

    ioserv.run();

}

If I replace the unix socket with a tcp connection, like below, it builds and runs:

using socket_t = boost::asio::ip::tcp::socket;
using next_layer_t = socket_t;
auto ip_address = boost::asio::ip::address::from_string("172.20.0.2");
auto port = boost::lexical_cast<std::uint16_t>(6379);
boost::asio::ip::tcp::endpoint end_point(ip_address, port);
socket_t socket(ioserv, end_point.protocol());
socket.connect(end_point);
basiliscos commented 4 years ago

socket_t sock(ioserv); bredis::Connection bredis_con(std::move(socket));

The local variable is named sock, while you are trying to move it as socket.

I found that naming variables as socket could cause a lot of issues because of socket(2) syscall on *nix, so it is better to avoid that name.

BarbBoyaji commented 4 years ago

Thank you, I appreciate the help! I changed the name and it's working as expected.