search

apache / pulsar-client-python

Apache Pulsar Python client library
https://pulsar.apache.org/
Apache License 2.0
51 stars 39 forks source link

[Bug] when pulsar python-client use tls connect pulsar then run coredump #136

Open liliang950210 opened 1 year ago

liliang950210 commented 1 year ago

Search before asking

Version

2.10.2

Minimal reproduce step

My os is eulerosv2r10.x86_64

$ ./python
Python 3.9.11 (tags/3.9.11-htrunk12-dirty:afe9e7d, Jan  1 1970, 00:00:01)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ssl.OPENSSL_VERSION
'OpenSSL 3.0.7 1 Nov 2022'

.. when i use python-client connect pulsar-server with TLS，the process coredump.

What did you expect to see?

Process run success and get data from pulsar.

What did you see instead?

coredump file -

#0  0x00007f20645c147a in bio_free ()
   from /xxx/infers/predict/python/venv/bms_data_receiver2/1.0.0/bms_data_receiver2/fa118b16ea8341d7b88772c8b7719863/_pulsar.cpython-39-x86_64-linux-gnu.so
[Current thread is 1 (LWP 218556)]
(gdb) bt
#0  0x00007f20645c147a in bio_free ()
   from /xxx/infers/predict/python/venv/bms_data_receiver2/1.0.0/bms_data_receiver2/fa118b16ea8341d7b88772c8b7719863/_pulsar.cpython-39-x86_64-linux-gnu.so
apache/pulsar#1  0x000000000069086a in BIO_ctrl ()
apache/pulsar#2  0x00007f20645c1e34 in BIO_new_bio_pair ()
   from /xxx/infers/predict/python/venv/bms_data_receiver2/1.0.0/bms_data_receiver2/fa118b16ea8341d7b88772c8b7719863/_pulsar.cpython-39-x86_64-linux-gnu.so
apache/pulsar#3  0x00007f20643d2272 in boost::asio::ssl::detail::engine::engine (context=<optimized out>, this=0x1f2c1c8)
    at /usr/local/include/boost/asio/ssl/detail/impl/engine.ipp:55
apache/pulsar#4  boost::asio::ssl::detail::stream_core::stream_core (this=0x1f2c1c8, context=<optimized out>, io_context=...)
    at /usr/local/include/boost/asio/ssl/detail/stream_core.hpp:48
apache/pulsar#5  0x00007f20643ca0d8 in boost::asio::ssl::stream<boost::asio::basic_stream_socket<boost::asio::ip::tcp>&>::stream<boost::asio::basic_stream_socket<boost::asio::ip::tcp>&> (ctx=..., arg=..., this=<optimized out>) at /usr/local/include/boost/asio/ssl/impl/context.ipp:378
apache/pulsar#6  pulsar::ExecutorService::createTlsSocket(std::shared_ptr<boost::asio::basic_stream_socket<boost::asio::ip::tcp> >&, boost::asio::ssl::context&) () at /pulsar/pulsar-client-cpp/lib/ExecutorService.cc:73
apache/pulsar#7  0x00007f20644c5c97 in pulsar::ClientConnection::ClientConnection (std::string const&, std::string const&, std::shared_ptr<pulsar::ExecutorService>, pulsar::ClientConfiguration const&, std::shared_ptr<pulsar::Authentication> const&) ()
    at /pulsar/pulsar-client-cpp/lib/ClientConnection.cc:251
apache/pulsar#8  0x00007f206439b57b in pulsar::ConnectionPool::getConnectionAsync(std::string const&, std::string const&) ()
    at /opt/rh/devtoolset-9/root/usr/include/c++/9/bits/shared_ptr_base.h:1020
apache/pulsar#9  0x00007f20644b043b in pulsar::BinaryProtoLookupService::getPartitionMetadataAsync(std::shared_ptr<pulsar::TopicName> const&) ()
    at /pulsar/pulsar-client-cpp/lib/BinaryProtoLookupService.cc:76
apache/pulsar#10 0x00007f20643818b7 in pulsar::ClientImpl::subscribeAsync(std::string const&, std::string const&, pulsar::ConsumerConfiguration const&, std::function<void (pulsar::Result, pulsar::Consumer)>) () at /opt/rh/devtoolset-9/root/usr/include/c++/9/bits/shared_ptr_base.h:1020
apache/pulsar#11 0x00007f2064376746 in pulsar::Client::subscribeAsync(std::string const&, std::string const&, pulsar::ConsumerConfiguration const&, std::function<void (pulsar::Result, pulsar::Consumer)>) () at /opt/rh/devtoolset-9/root/usr/include/c++/9/bits/shared_ptr_base.h:1020
apache/pulsar#12 0x00007f2064376e92 in pulsar::Client::subscribe(std::string const&, std::string const&, pulsar::ConsumerConfiguration const&, pulsar::Consumer&) () at /opt/rh/devtoolset-9/root/usr/include/c++/9/new:174
apache/pulsar#13 0x00007f2064347ece in Client_subscribe(pulsar::Client&, std::string const&, std::string const&, pulsar::ConsumerConfiguration const&) () at /pulsar/pulsar-client-cpp/python/src/client.cc:37
apache/pulsar#14 0x00007f206434a13e in boost::python::detail::invoke<boost::python::to_python_value<pulsar::Consumer const&>, pulsar::Consumer (*)(pul---Type <return> to continue, or q <return> to quit---
sar::Client&, std::string const&, std::string const&, pulsar::ConsumerConfiguration const&), boost::python::arg_from_python<pulsar::Client&>, boost::python::arg_from_python<std::string const&>, boost::python::arg_from_python<std::string const&>, boost::python::arg_from_python<pulsar::ConsumerConfiguration const&> > (ac3=..., ac2=..., ac1=..., ac0=<synthetic pointer>..., f=<optimized out>, rc=...)
    at /usr/local/include/boost/python/converter/arg_from_python.hpp:213
apache/pulsar#15 boost::python::detail::caller_arity<4u>::impl<pulsar::Consumer (*)(pulsar::Client&, std::string const&, std::string const&, pulsar::ConsumerConfiguration const&), boost::python::default_call_policies, boost::mpl::vector5<pulsar::Consumer, pulsar::Client&, std::string const&, std::string const&, pulsar::ConsumerConfiguration const&> >::operator() (args_=<optimized out>, this=<optimized out>)
    at /usr/local/include/boost/python/detail/caller.hpp:216
apache/pulsar#16 boost::python::objects::caller_py_function_impl<boost::python::detail::caller<pulsar::Consumer (*)(pulsar::Client&, std::string const&, std::string const&, pulsar::ConsumerConfiguration const&), boost::python::default_call_policies, boost::mpl::vector5<pulsar::Consumer, pulsar::Client&, std::string const&, std::string const&, pulsar::ConsumerConfiguration const&> > >::operator() (this=<optimized out>, 
    args=<optimized out>, kw=<optimized out>) at /usr/local/include/boost/python/object/py_function.hpp:38
apache/pulsar#17 0x00007f206450479b in boost::python::objects::function::call(_object*, _object*) const ()
   from /xxx/infers/predict/python/venv/bms_data_receiver2/1.0.0/bms_data_receiver2/fa118b16ea8341d7b88772c8b7719863/_pulsar.cpython-39-x86_64-linux-gnu.so
apache/pulsar#18 0x00007f2064504a38 in boost::detail::function::void_function_ref_invoker0<boost::python::objects::(anonymous namespace)::bind_return, void>::invoke(boost::detail::function::function_buffer&) ()
   from /xxx/infers/predict/python/venv/bms_data_receiver2/1.0.0/bms_data_receiver2/fa118b16ea8341d7b88772c8b7719863/_pulsar.cpython-39-x86_64-linux-gnu.so
apache/pulsar#19 0x00007f20645096e3 in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const ()
   from /xxx/infers/predict/python/venv/bms_data_receiver2/1.0.0/bms_data_receiver2/fa118b16ea8341d7b88772c8b7719863/_pulsar.cpython-39-x86_64-linux-gnu.so
apache/pulsar#20 0x00007f2064330ec8 in boost::python::detail::translate_exception<PulsarException, void (*)(PulsarException const&)>::operator() (
    translate=0x7f2064330d10 <translateException(PulsarException const&)>, f=..., handler=..., this=<optimized out>)
    at /usr/local/include/boost/python/detail/translate_exception.hpp:39
apache/pulsar#21 boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(PulsarException const&)> >::operator()<bool, boost::python::detail::translate_exception<PulsarException, void (*)(PulsarException const&)>, boost::_bi::rrlist2<boost::python::detail::exception_handler const&, boost::function0<void> const&> > (a=<synthetic pointer>..., f=..., this=<optimized out>)
    at /usr/local/include/boost/bind/bind.hpp:388
apache/pulsar#22 boost::_bi::bind_t<bool, boost::python::detail::translate_exception<PulsarException, void (*)(PulsarException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(PulsarException const&)> > >::operator()<boost::python::detail::exception_h---Type <return> to continue, or q <return> to quit---
andler const&, boost::function0<void> const&> (a2=..., a1=..., this=<optimized out>) at /usr/local/include/boost/bind/bind.hpp:1318
apache/pulsar#23 boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<PulsarException, void (*)(PulsarException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(PulsarException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke (function_obj_ptr=..., a0=..., a1=...)
    at /usr/local/include/boost/function/function_template.hpp:138
apache/pulsar#24 0x00007f206450962f in boost::python::handle_exception_impl(boost::function0<void>) ()
   from /xxx/infers/predict/python/venv/bms_data_receiver2/1.0.0/bms_data_receiver2/fa118b16ea8341d7b88772c8b7719863/_pulsar.cpython-39-x86_64-linux-gnu.so
apache/pulsar#25 0x00007f2064502302 in function_call ()
   from /xxx/infers/predict/python/venv/bms_data_receiver2/1.0.0/bms_data_receiver2/fa118b16ea8341d7b88772c8b7719863/_pulsar.cpython-39-x86_64-linux-gnu.so
apache/pulsar#26 0x00000000004b32fa in _PyObject_MakeTpCall (tstate=0x1ce08f0, callable=0x1e37440, args=<optimized out>, nargs=4, keywords=0x0)
    at Objects/call.c:191
apache/pulsar#27 0x000000000061c175 in _PyObject_VectorcallTstate (kwnames=<optimized out>, nargsf=<optimized out>, args=0x1e46e00, 
    callable=0x1e37440, tstate=0x1ce08f0) at ./Include/cpython/abstract.h:116
apache/pulsar#28 method_vectorcall (method=<optimized out>, args=0x1e46e08, nargsf=<optimized out>, kwnames=<optimized out>)
    at Objects/classobject.c:53
apache/pulsar#29 0x00000000004a473e in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, args=0x1e46e08, callable=0x7f2064a78240, 
    tstate=0x1ce08f0) at ./Include/cpython/abstract.h:118
apache/pulsar#30 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x1e46e08, callable=0x7f2064a78240)
    at ./Include/cpython/abstract.h:127
apache/pulsar#31 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x1ce08f0) at Python/ceval.c:5077
apache/pulsar#32 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x1e46be0, throwflag=<optimized out>) at Python/ceval.c:3489
apache/pulsar#33 0x00000000005381ae in _PyEval_EvalFrame (throwflag=0, f=0x1e46be0, tstate=0x1ce08f0) at ./Include/internal/pycore_ceval.h:40
apache/pulsar#34 _PyEval_EvalCode (tstate=0x1ce08f0, _co=<optimized out>, globals=<optimized out>, locals=locals@entry=0x0, args=<optimized out>, 
    argcount=1, kwnames=0x7f206382d328, kwargs=0x7f2065126df0, kwcount=4, kwstep=1, defs=0x7f20650d5f58, defcount=15, kwdefs=0x0, 
    closure=0x0, name=0x7f2064e190b0, qualname=0x7f2064e18530) at Python/ceval.c:4329
apache/pulsar#35 0x00000000004b30e1 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>, nargsf=<optimized out>, 
    kwnames=<optimized out>) at Objects/call.c:396
apache/pulsar#36 0x000000000061c103 in _PyObject_VectorcallTstate (kwnames=<optimized out>, nargsf=1, args=0x7f2065126de8, callable=0x7f2062aec9d0, 
    tstate=0x1ce08f0) at ./Include/cpython/abstract.h:118
---Type <return> to continue, or q <return> to quit---
apache/pulsar#37 method_vectorcall (method=<optimized out>, args=0x7f2065126df0, nargsf=<optimized out>, kwnames=<optimized out>)
    at Objects/classobject.c:53
apache/pulsar#38 0x00000000004a3228 in _PyObject_VectorcallTstate (kwnames=0x7f206382d310, nargsf=<optimized out>, args=0x7f2065126df0, 
    callable=0x7f2064e14800, tstate=0x1ce08f0) at ./Include/cpython/abstract.h:118
apache/pulsar#39 PyObject_Vectorcall (kwnames=0x7f206382d310, nargsf=<optimized out>, args=0x7f2065126df0, callable=0x7f2064e14800)
    at ./Include/cpython/abstract.h:127
apache/pulsar#40 call_function (kwnames=0x7f206382d310, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x1ce08f0) at Python/ceval.c:5077
apache/pulsar#41 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x7f2065126c80, throwflag=<optimized out>) at Python/ceval.c:3537
apache/pulsar#42 0x00000000005381ae in _PyEval_EvalFrame (throwflag=0, f=0x7f2065126c80, tstate=0x1ce08f0) at ./Include/internal/pycore_ceval.h:40
apache/pulsar#43 _PyEval_EvalCode (tstate=0x1ce08f0, _co=_co@entry=0x7f2064e12a80, globals=globals@entry=0x7f20651cc3c0, 
    locals=locals@entry=0x7f20651cc3c0, args=args@entry=0x0, argcount=argcount@entry=0, kwnames=0x0, kwargs=0x0, kwcount=0, kwstep=2, 
    defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x0, qualname=0x0) at Python/ceval.c:4329
apache/pulsar#44 0x0000000000538542 in _PyEval_EvalCodeWithName (_co=_co@entry=0x7f2064e12a80, globals=globals@entry=0x7f20651cc3c0, 
    locals=locals@entry=0x7f20651cc3c0, args=args@entry=0x0, argcount=argcount@entry=0, kwnames=kwnames@entry=0x0, kwargs=0x0, 
    kwcount=0, kwstep=2, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x0, qualname=0x0) at Python/ceval.c:4361
apache/pulsar#45 0x000000000053858e in PyEval_EvalCodeEx (_co=_co@entry=0x7f2064e12a80, globals=globals@entry=0x7f20651cc3c0, 
    locals=locals@entry=0x7f20651cc3c0, args=args@entry=0x0, argcount=argcount@entry=0, kws=kws@entry=0x0, kwcount=0, defs=0x0, 
    defcount=0, kwdefs=0x0, closure=0x0) at Python/ceval.c:4377
apache/pulsar#46 0x00000000005385bb in PyEval_EvalCode (co=co@entry=0x7f2064e12a80, globals=globals@entry=0x7f20651cc3c0, 
    locals=locals@entry=0x7f20651cc3c0) at Python/ceval.c:828
apache/pulsar#47 0x0000000000569cc9 in run_eval_code_obj (tstate=tstate@entry=0x1ce08f0, co=co@entry=0x7f2064e12a80, 
    globals=globals@entry=0x7f20651cc3c0, locals=locals@entry=0x7f20651cc3c0) at Python/pythonrun.c:1221
apache/pulsar#48 0x0000000000569dc1 in run_mod (mod=mod@entry=0x1ddd2f0, filename=filename@entry=0x7f2065156ab0, globals=0x7f20651cc3c0, 
    locals=0x7f20651cc3c0, flags=flags@entry=0x7ffc592d1188, arena=arena@entry=0x7f2064e33c10) at Python/pythonrun.c:1242
apache/pulsar#49 0x000000000056c428 in PyRun_InteractiveOneObjectEx (fp=fp@entry=0x7f2072171860 <_IO_2_1_stdin_>, 
    filename=filename@entry=0x7f2065156ab0, flags=flags@entry=0x7ffc592d1188) at Python/pythonrun.c:274
apache/pulsar#50 0x000000000056c6f6 in PyRun_InteractiveLoopFlags (fp=fp@entry=0x7f2072171860 <_IO_2_1_stdin_>, 
    filename_str=filename_str@entry=0xa279f2 "<stdin>", flags=flags@entry=0x7ffc592d1188) at Python/pythonrun.c:127
apache/pulsar#51 0x000000000056cf2e in PyRun_AnyFileExFlags (fp=0x7f2072171860 <_IO_2_1_stdin_>, filename=filename@entry=0xa279f2 "<stdin>", 
    closeit=closeit@entry=0, flags=flags@entry=0x7ffc592d1188) at Python/pythonrun.c:86
apache/pulsar#52 0x00000000004a8182 in pymain_run_stdin (cf=0x7ffc592d1188, config=0x1cdeea0) at Modules/main.c:512
---Type <return> to continue, or q <return> to quit---
apache/pulsar#53 pymain_run_python (exitcode=0x7ffc592d1180) at Modules/main.c:601
apache/pulsar#54 Py_RunMain () at Modules/main.c:677
apache/pulsar#55 0x00000000004a83f6 in pymain_main (args=args@entry=0x7ffc592d1260) at Modules/main.c:707
apache/pulsar#56 0x00000000004a84d9 in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:731
apache/pulsar#57 0x00007f2071fe5c87 in __libc_start_main () from /usr/lib64/libc.so.6
apache/pulsar#58 0x00000000004a717c in _start ()

Anything else?

No response

Are you willing to submit a PR?

tisonkun commented 1 year ago

@liliang950210

  1. The issue doesn't contain a reproduce how you meet the coredump.
  2. You may try out the new version of pulsar-python-client like 3.2.0. It should be compatible with any broker in previous versions (otherwise, it's a bug to be fixed).
BewareMyPower commented 1 year ago

You can try the latest Python client, which might fix this issue. There is something wrong with the previous linking approach before 3.0.0.