search

Thriftpy / thriftpy2

Pure python approach of Apache Thrift.
MIT License
572 stars 91 forks source link

连接超时后，将超时当次的thrift server结果覆盖给下次正常访问的结果 #130

Closed accjiyun closed 4 years ago

accjiyun commented 4 years ago

版本号：0.4.10

当thrift client连接时间超过设置的超时阈值后，连接并没有断开，而是将本次的超时结果数据放到了缓存中，并且覆盖了下次访问thrift server的返回结果。

举个例子： thrift server:

def get(msg):
    return msg

正常情况下，依次访问返回结果如下： client.get(1) 得到的结果是1 client.get(2) 得到的结果是2

如果在第一步client.get(1) 超时返回结果为空，第二次访问client.get(2) 得到的是1

我的连接方式和超时错误如下：

            self._client = make_client(getattr(self.service_thrift, self.service),
                                       host, int(port),
                                       trans_factory=TFramedTransportFactory(),
                                       timeout=100
                                       )
    fname, mtype, rseqid = self._iprot.read_message_begin()
  File "thriftpy2/protocol/cybin/cybin.pyx", line 429, in cybin.TCyBinaryProtocol.read_message_begin
    size = read_i32(self.trans)
  File "thriftpy2/protocol/cybin/cybin.pyx", line 60, in cybin.read_i32
    buf.c_read(4, data)
  File "thriftpy2/transport/framed/cyframed.pyx", line 48, in thriftpy2.transport.framed.cyframed.TCyFramedTransport.c_read
    self.read_frame()
  File "thriftpy2/transport/framed/cyframed.pyx", line 68, in thriftpy2.transport.framed.cyframed.TCyFramedTransport.read_frame
    self.read_trans(4, frame_len)
  File "thriftpy2/transport/framed/cyframed.pyx", line 31, in thriftpy2.transport.framed.cyframed.TCyFramedTransport.read_trans
    cdef int i = self.rbuf.read_trans(self.trans, sz, out)
  File "thriftpy2/transport/cybase.pyx", line 61, in thriftpy2.transport.cybase.TCyBuffer.read_trans
    new_data = trans.read(cap)
  File "/home/jiyun/.pyenv/versions/XXX/lib/python2.7/site-packages/thriftpy2/transport/socket.py", line 110, in read
    buff = self.sock.recv(sz)
timeout: timed out
ethe commented 4 years ago

Use thrift_connector to resolve this problem, or if existing client timeout, initialize a client again. A client instance always uses bonded transport (socket) and it would not remove it even it is timeout.

accjiyun commented 4 years ago

@ethe thrift_connector 接口似乎不太友好，ClientPool() 不方便直接指定transport和protocol

class ThriftPyTFramedClient(ThriftPyBaseClient):
    @classmethod
    def get_protoco_factory(self):
        from thriftpy2.protocol import TBinaryProtocolFactory
        return TBinaryProtocolFactory().get_protocol

    @classmethod
    def get_transport_factory(self):
        from thriftpy2.transport import TFramedTransportFactory
        return TFramedTransportFactory().get_transport

pool = connection_pool.ClientPool(
                getattr(self.service_thrift, self.service),
                host, int(port), 1000,
                connection_class=ThriftPyTFramedClient
            )
ethe commented 4 years ago

Yes, for now, it only supports define a subclass to ThriftPyBaseClient. I will consider to improve it.