QuincyChen / baserver

Automatically exported from code.google.com/p/baserver
0 stars 0 forks source link

刚刚看了下代码, 我觉得存在一点问题 #3

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago
因为代码中
server_handler中do_read, do_write会往worker thread 
中post回调,而example中
是on_read中回调了async_write那么有中情况会出现问题, 
就是读的包远小于写的
包。因为asio就是一次写还没回调,结果因为异步读调用了下�
��次的异步写,后一次
的异步写把前一次的没有完成的异步写,在asio buffer 
中给覆盖了。 只是看的代
码,没有时间实践,有问题可以继续讨论, 
您也可写一个case测试下

Original issue reported on code.google.com by kofsho...@gmail.com on 5 Jul 2009 at 5:44

GoogleCodeExporter commented 9 years ago
在on_read和on_write中调用async_write或async_read应该是没有问题的��
�首
先,read和write的buffer是独立、分开的,框架提供了读/写io_buff
er,
用户也可以自行提供、管理和实现buffer;其次,on_read和on_writ
e执
行时已经完成了上一次的read和write,在里面调用async_write等也
没
有问题,asio会post给io_service,按照顺序执行。bas代码的一个��
�心
概念就是由io_service管理异步操作的同步,而不是通过用户锁�
��
方式。examples中的主要代码我通过多次测试,从工作表现来看
应该是正常的。

Original comment by moore...@gmail.com on 6 Jul 2009 at 6:12

GoogleCodeExporter commented 9 years ago
最近忙,等忙完看看,此外我觉得server_work的on_write回调怎么�
��有写的字节数,假设这样的
场景:
class my_server_work
        public OtherInterface,
        public boost::enable_shared_from_this<my_server_work>
{
   on_read(...) ...
   on_write(...)
   {
       m_phandler->write_buffer().consume(<byte_transfered>);
   }

   ...

   on_producedatacallback(...)
   {
       ByteArray buf;
       genPackage(buf, ...)
       bas::io_buffer& pbuffer = m_phandler->write_buffer();
       unsigned char* pbuf = pbuffer.data();
       memcpy(pbuf, &buf[0], buf.size());
       pbuffer.produce(buf.size());
       m_phandler->async_write(buf.size());
   }

private:
   server_handler_type* m_phandler;

};

比如这样子的work,见尖括号的地方

Original comment by kofsho...@gmail.com on 15 Jul 2009 at 3:53

GoogleCodeExporter commented 9 years ago
代码可能语法不通,不要介意

Original comment by kofsho...@gmail.com on 15 Jul 2009 at 3:54

GoogleCodeExporter commented 9 years ago
因为异步写操作处理比较简单,以前忽略了,现在更新到
0,30.6,对写on_write回调增加了成功写的字节数参数。

Original comment by moore...@gmail.com on 22 Jul 2009 at 8:24

GoogleCodeExporter commented 9 years ago
kofshower, 谢谢

Original comment by moore...@gmail.com on 22 Jul 2009 at 8:25

GoogleCodeExporter commented 9 years ago
我瞅到,service_handler中read_buffer 
是一个公有的接口,而且没有线程同步。而且比如
async_read_some(boost::asio::buffer(read_buffer().data() + 
read_buffer().size(), 
read_buffer().space())),没有检查space。其实我建议,on_read应该是
on_read(service_handler, std::vector<unsigned 
char>&)或是on_read(service_handler, 
std::vector<unsigned char>&, size_t transferred) 
第一种是少了一个参数,第二种是里面可以判
断大小是否够用然后重新resize,可以提高一些效率。此外必��
�要判断space(),不够则执行crunch()。
space()仍然为0那么close_i

Original comment by kofsho...@gmail.com on 29 Jul 2009 at 1:25

GoogleCodeExporter commented 9 years ago
代码

Original comment by kofsho...@gmail.com on 29 Jul 2009 at 2:58

Attachments:

GoogleCodeExporter commented 9 years ago
writebuffer可以自己写一些deque 
来append要写的数据包,不需要io_buffer,优点是人为的控制asio�
��
写队列

Original comment by kofsho...@gmail.com on 29 Jul 2009 at 3:00

GoogleCodeExporter commented 9 years ago
修改了write_buffer,必须这样写或是用strand来串行写

Original comment by kofsho...@gmail.com on 29 Jul 2009 at 3:40

Attachments:

GoogleCodeExporter commented 9 years ago
基本上改好了,由于我认为最好不要在服务逻辑中写buffer,��
�把这个东西的自由改由使用者实现了
,之前提交的service_handler没有测试过,应该少写了iobuffer的pro
duce之类的。现在这个文
件经过测试了,可以work,由用户提供buffer的实现

Original comment by junjunc...@gmail.com on 31 Jul 2009 at 3:53

Attachments:

GoogleCodeExporter commented 9 years ago
kofshower,
你可能没有注意,server_handler是不需要加锁的,线程同步
由io_service的同步机制来保障;
另外,io_buffer是一个缺省实现的简单缓冲区,供简单应用使��
�,
通过参数可以将大小置为零;如果用户需要比较复杂的缓冲��
�
管理,可以自行实现后,调用async_read(boost::asio::mutable_buffers_1
 buffers)
和async_write(boost::asio::mutable_buffers_1 buffers)来完成读写操作。

Original comment by moore...@gmail.com on 3 Aug 2009 at 4:41

GoogleCodeExporter commented 9 years ago
我就是io_buffer 觉得不怎么顺手,然后只留下async_write 
async_read async_read_some 以
bytearray为参数

Original comment by junjunc...@gmail.com on 4 Aug 2009 at 1:12

GoogleCodeExporter commented 9 years ago
kofshower,
你可能没有注意,server_handler是不需要加锁的,线程同步
由io_service的同步机制来保障;
这一点我不同意,我测试了代码,问题恰好发生在这儿,如��
�在client_work里面同时发起多次
service_handler的异步调用。你就知道我的写法的意义了。我写��
�一个测试工程,重现了会收到
错误包的情况。用cmake作为编译系统。需要有zlib,log4cxx,boos
t 1_35, tinyxml这几个库。

Original comment by junjunc...@gmail.com on 5 Aug 2009 at 9:26

Attachments:

GoogleCodeExporter commented 9 years ago
关键是这种情况:
void on_write(client_handler_type& handler, size_t bytesOfTransfer)
        {
           for(std::size_t i = 0; i < 5000; ++i)
           {
               specificrequire(handler, NET_CMD_TESTBAS);
           }
        }

Original comment by junjunc...@gmail.com on 5 Aug 2009 at 9:29

GoogleCodeExporter commented 9 years ago
这里可能有一个出发点的问题。
bas设计的初衷是不管IO如何异步,但逻辑处理流程是同步
的。即针对每一次IO,系统要做出同步响应。异步只不过实
现IO与逻辑处理隔离、提高IO效率的手段。
其实,前述代码on_write中的5000次循环可以理解为一次IO,即
只调用一次async_write,http_server中就有类似的实现,如
std::vector<boost::asio::const_buffer> reply::to_buffers()
{
  std::vector<boost::asio::const_buffer> buffers;
  buffers.push_back(status_strings::to_buffer(status));
  for (std::size_t i = 0; i < headers.size(); ++i)
  {
    header& h = headers[i];
    buffers.push_back(boost::asio::buffer(h.name));
    buffers.push_back(boost::asio::buffer(misc_strings::name_value_separator));
    buffers.push_back(boost::asio::buffer(h.value));
    buffers.push_back(boost::asio::buffer(misc_strings::crlf));
  }
  buffers.push_back(boost::asio::buffer(misc_strings::crlf));
  buffers.push_back(boost::asio::buffer(content));
  return buffers;
}
然后调用handler.async_write(reply_.to_buffers());
当然,如果要在一次IO中发起多个异步操作,bas目前是没有
实现的,但是从另一个角度讲,多个没有前后以来关系的IO
其实可组合成一个IO。

Original comment by moore...@gmail.com on 6 Aug 2009 at 2:18

GoogleCodeExporter commented 9 years ago
sorry,应该是没有前后依赖关系的IO其实可组合成一个IO。

Original comment by moore...@gmail.com on 6 Aug 2009 at 2:19

GoogleCodeExporter commented 9 years ago
恩,明白。可是那个async_write在boost1.35,gcc4.3.2下面编译不了�
��
client_work.hpp:48: error: no matching function for call to
‘bas::service_handler<bastest::client_work,
boost::asio::basic_stream_socket<boost::asio::ip::tcp,
boost::asio::stream_socket_service<boost::asio::ip::tcp> >
>::async_write(std::vector<boost::asio::const_buffer,
std::allocator<boost::asio::const_buffer> >&)’
/home/bmc/SzReceiver/stquotereceiver/test/testbas/include/bas/service_handler.hp
p:233: note:
candidates are: void bas::service_handler<Work_Handler,
Socket_Service>::async_write() [with Work_Handler = bastest::client_work,
Socket_Service = boost::asio::basic_stream_socket<boost::asio::ip::tcp,
boost::asio::stream_socket_service<boost::asio::ip::tcp> >]
/home/bmc/SzReceiver/stquotereceiver/test/testbas/include/bas/service_handler.hp
p:241: note:
                void bas::service_handler<Work_Handler,
Socket_Service>::async_write(size_t) [with Work_Handler = bastest::client_work,
Socket_Service = boost::asio::basic_stream_socket<boost::asio::ip::tcp,
boost::asio::stream_socket_service<boost::asio::ip::tcp> >]
/home/bmc/SzReceiver/stquotereceiver/test/testbas/include/bas/service_handler.hp
p:250: note:
                void bas::service_handler<Work_Handler,
Socket_Service>::async_write(boost::asio::mutable_buffers_1) [with Work_Handler 
=
bastest::client_work, Socket_Service =
boost::asio::basic_stream_socket<boost::asio::ip::tcp,
boost::asio::stream_socket_service<boost::asio::ip::tcp> >]

Original comment by kofsho...@gmail.com on 7 Aug 2009 at 8:58

GoogleCodeExporter commented 9 years ago
client_work.hpp:48: error: no matching function for call to
‘bas::service_handler<bastest::client_work,
boost::asio::basic_stream_socket<boost::asio::ip::tcp,
boost::asio::stream_socket_service<boost::asio::ip::tcp> >
>::async_write(std::vector<boost::asio::const_buffer,
std::allocator<boost::asio::const_buffer> >&)’
可能是你改过的,原本的bas没有这种调用方式

Original comment by moore...@gmail.com on 10 Aug 2009 at 7:48

GoogleCodeExporter commented 9 years ago
可能你没有在linux下面跑过example。下面http那个例子也是一样�
��错:
Scanning dependencies of target testbas
make[2]: Leaving directory `/home/bmc/Source/chaijunjun/testbas/release'
make -f CMakeFiles/testbas.dir/build.make CMakeFiles/testbas.dir/build
make[2]: Entering directory `/home/bmc/Source/chaijunjun/testbas/release'
/usr/bin/cmake -E cmake_progress_report
/home/bmc/Source/chaijunjun/testbas/release/CMakeFiles 1
[ 16%] Building CXX object CMakeFiles/testbas.dir/src/posix_main_server.cpp.o
/usr/bin/g++    -O3 -Wall -Wno-deprecated -ftemplate-depth-50 -fPIC -fno-inline
-I/home/bmc/Source/chaijunjun/testbas/include   -o
CMakeFiles/testbas.dir/src/posix_main_server.cpp.o -c
/home/bmc/Source/chaijunjun/testbas/src/posix_main_server.cpp
In file included from 
/home/bmc/Source/chaijunjun/testbas/src/posix_main_server.cpp:21:
/home/bmc/Source/chaijunjun/testbas/include/server_work.hpp: In member function 
‘void
http::server::server_work::on_read(bas::service_handler<http::server::server_wor
k,
boost::asio::basic_stream_socket<boost::asio::ip::tcp,
boost::asio::stream_socket_service<boost::asio::ip::tcp> > >&, size_t)’:
/home/bmc/Source/chaijunjun/testbas/include/server_work.hpp:62: error: no 
matching
function for call to ‘bas::service_handler<http::server::server_work,
boost::asio::basic_stream_socket<boost::asio::ip::tcp,
boost::asio::stream_socket_service<boost::asio::ip::tcp> >
>::async_write(std::vector<boost::asio::const_buffer,
std::allocator<boost::asio::const_buffer> >)’
/home/bmc/Source/chaijunjun/testbas/include/bas/service_handler.hpp:228: note:
candidates are: void bas::service_handler<Work_Handler,
Socket_Service>::async_write() [with Work_Handler = http::server::server_work,
Socket_Service = boost::asio::basic_stream_socket<boost::asio::ip::tcp,
boost::asio::stream_socket_service<boost::asio::ip::tcp> >]
/home/bmc/Source/chaijunjun/testbas/include/bas/service_handler.hpp:236: note:  

          void bas::service_handler<Work_Handler,
Socket_Service>::async_write(size_t) [with Work_Handler = 
http::server::server_work,
Socket_Service = boost::asio::basic_stream_socket<boost::asio::ip::tcp,
boost::asio::stream_socket_service<boost::asio::ip::tcp> >]
/home/bmc/Source/chaijunjun/testbas/include/bas/service_handler.hpp:245: note:  

          void bas::service_handler<Work_Handler,
Socket_Service>::async_write(boost::asio::mutable_buffers_1) [with Work_Handler 
=
http::server::server_work, Socket_Service =
boost::asio::basic_stream_socket<boost::asio::ip::tcp,
boost::asio::stream_socket_service<boost::asio::ip::tcp> >]
/home/bmc/Source/chaijunjun/testbas/include/server_work.hpp:67: error: no 
matching
function for call to ‘bas::service_handler<http::server::server_work,
boost::asio::basic_stream_socket<boost::asio::ip::tcp,
boost::asio::stream_socket_service<boost::asio::ip::tcp> >
>::async_write(std::vector<boost::asio::const_buffer,
std::allocator<boost::asio::const_buffer> >)’
/home/bmc/Source/chaijunjun/testbas/include/bas/service_handler.hpp:228: note:
candidates are: void bas::service_handler<Work_Handler,
Socket_Service>::async_write() [with Work_Handler = http::server::server_work,
Socket_Service = boost::asio::basic_stream_socket<boost::asio::ip::tcp,
boost::asio::stream_socket_service<boost::asio::ip::tcp> >]
/home/bmc/Source/chaijunjun/testbas/include/bas/service_handler.hpp:236: note:  

          void bas::service_handler<Work_Handler,
Socket_Service>::async_write(size_t) [with Work_Handler = 
http::server::server_work,
Socket_Service = boost::asio::basic_stream_socket<boost::asio::ip::tcp,
boost::asio::stream_socket_service<boost::asio::ip::tcp> >]
/home/bmc/Source/chaijunjun/testbas/include/bas/service_handler.hpp:245: note:  

          void bas::service_handler<Work_Handler,
Socket_Service>::async_write(boost::asio::mutable_buffers_1) [with Work_Handler 
=
http::server::server_work, Socket_Service =
boost::asio::basic_stream_socket<boost::asio::ip::tcp,
boost::asio::stream_socket_service<boost::asio::ip::tcp> >]
/home/bmc/Source/chaijunjun/testbas/include/server_work.hpp:72: error: 
‘clear’ was
not declared in this scope
make[2]: *** [CMakeFiles/testbas.dir/src/posix_main_server.cpp.o] Error 1
make[2]: Leaving directory `/home/bmc/Source/chaijunjun/testbas/release'
make[1]: *** [CMakeFiles/testbas.dir/all] Error 2
make[1]: Leaving directory `/home/bmc/Source/chaijunjun/testbas/release'
make: *** [all] Error 2

Original comment by kofsho...@gmail.com on 10 Aug 2009 at 9:15

GoogleCodeExporter commented 9 years ago
sorry,当时在linux下面编译确实出现了一下问题,对gcc不是太
了解,gcc好像对template的支持不如vc和icc,所以对下面代码部
分作了一些调整,当时只编译echo_server通过后就没有编译其
它的examples,这个问题我要看看怎么调整比较好。
  /// Start an asynchronous operation from any thread to write buffers to the socket.
#if defined(__GNUC__)
  void async_write(boost::asio::mutable_buffers_1 buffers)
  {
    io_service().dispatch(boost::bind(&service_handler_type::async_write_1,
        shared_from_this(),
        buffers));
  }
#else  // other c++ compiler
  template<typename Buffers>
  void async_write(const Buffers& buffers)
  {
    io_service().dispatch(boost::bind(&service_handler_type::async_write_i<Buffers>,
        shared_from_this(),
        buffers));
  }
#endif
另外还有一个原因就是部分接口与最初版本相比作了一些调
整,如clear改称了on_clear等。

Original comment by moore...@gmail.com on 11 Aug 2009 at 1:26

GoogleCodeExporter commented 9 years ago
这个简单 
最简单的做法是加一个重载就行了,要不然写一个类似consumin
g_buffer的也行
比如 void async_write(const std::vector<const_buffer>& buffers)

Original comment by kofsho...@gmail.com on 13 Aug 2009 at 4:19

GoogleCodeExporter commented 9 years ago
我知道,不过觉得有点丑陋而已,而且我也不确定与bind是否�
��
关,我想看看迟点boost发布1.4后能否有所改变。

Original comment by moore...@gmail.com on 17 Aug 2009 at 7:31

GoogleCodeExporter commented 9 years ago
service_handler_pool的实现是不是有问题啊?
首先server.h里的get_service_handler调用没有进行同步,client.h里倒
是有,不知道为啥认为server端不需要传入一个mutex.

其次,get_service_handler里的实现也太简单了,if 
(service_handler.get() == 
0)就马上push_back,这样的话,如果运行个1年,我估计这个service
_handler_pool不断的增加直至内存耗尽.
不知道有没有改进?

Original comment by ooseven1...@gmail.com on 8 Aug 2011 at 2:20

GoogleCodeExporter commented 9 years ago
更新了0.50版本,优化了service_handler_pool的动态管理

Original comment by moore...@gmail.com on 9 Oct 2011 at 7:11