search

TarsCloud / Tars

Tars is a high-performance RPC framework based on name service and Tars protocol, also integrated administration platform, and implemented hosting-service via flexible schedule.
BSD 3-Clause "New" or "Revised" License
9.8k stars 2.08k forks source link

Tars服务启动tarsproperty服务异常并产生很多core文件 #381

Closed jackylee92 closed 5 years ago

jackylee92 commented 5 years ago

What language are you using?

php

What operating system (Linux, Ubuntu, …) and version?

linux

What runtime / compiler are you using (e.g. jdk version or version of gcc)

gcc 版本 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC)

Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs). tars所有服务正常启动后，两三分钟后tarsproperty 结果变为 [alarm] down,pid not exist 并且进程ID改变，服务器 /data/tars/app_log/中产生一些 core 文件 image

image

jianhuihi commented 5 years ago

出现同样问题

jianhuihi commented 5 years ago 
[root@localhost app_log]# gdb /usr/local/app/tars/tarsnode/data/tars.tarsproperty/bin/tarsproperty core.32
core.32180  core.32452  core.32555  core.32582  
[root@localhost app_log]# gdb /usr/local/app/tars/tarsnode/data/tars.tarsproperty/bin/tarsproperty core.32
core.32180  core.32452  core.32555  core.32582  
[root@localhost app_log]# gdb /usr/local/app/tars/tarsnode/data/tars.tarsproperty/bin/tarsproperty core.32582 
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-114.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/local/app/tars/tarsnode/data/tars.tarsproperty/bin/tarsproperty...done.
[New LWP 32595]
[New LWP 32592]
[New LWP 32587]
[New LWP 32582]
[New LWP 32585]
[New LWP 32586]
[New LWP 32600]
[New LWP 32599]
[New LWP 32589]
[New LWP 32601]
[New LWP 32593]
[New LWP 32594]
[New LWP 32591]
[New LWP 32584]
[New LWP 32598]
[New LWP 32597]
[New LWP 32583]
[New LWP 32590]
[New LWP 32596]
[New LWP 32588]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/usr/local/app/tars/tarsnode/data/tars.tarsproperty/bin/tarsproperty --config=/'.
Program terminated with signal 11, Segmentation fault.
#0  tars::TC_MemChunk::allocate (this=0x1fd51a0) at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_mem_chunk.cpp:88
88          _pHead->_firstAvailableBlock = *((size_t *)result);
Missing separate debuginfos, use: debuginfo-install glibc-2.17-260.el7_6.3.x86_64 libgcc-4.8.5-36.el7.x86_64 libstdc++-4.8.5-36.el7.x86_64 zlib-1.2.7-18.el7.x86_64
(gdb) bt
#0  tars::TC_MemChunk::allocate (this=0x1fd51a0) at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_mem_chunk.cpp:88
#1  0x00000000005338e1 in allocate2 (iIndex=<synthetic pointer>, this=<optimized out>) at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_mem_chunk.cpp:99
#2  allocate2 (iIndex=<synthetic pointer>, this=<optimized out>) at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_mem_chunk.cpp:246
#3  tars::TC_MemMultiChunkAllocator::allocate2 (this=0x1fd43f0, iNeedSize=162, iAllocSize=@0x7fb2c4ff7418: 162, iIndex=@0x7fb2c4ff7388: 0)
    at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_mem_chunk.cpp:554
#4  0x000000000053397e in tars::TC_MemMultiChunkAllocator::allocate (this=<optimized out>, iNeedSize=<optimized out>, iAllocSize=@0x7fb2c4ff7418: 162)
    at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_mem_chunk.cpp:540
#5  0x000000000052a6c2 in tars::TC_HashMap::BlockAllocator::allocateMemBlock (this=0x1fd43d0, index=index@entry=324881, iAllocSize=@0x7fb2c4ff7418: 162, 
    vtData=std::vector of length 0, capacity 0) at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_hashmap.cpp:857
#6  0x000000000052b2b3 in tars::TC_HashMap::set (this=this@entry=0x1fd4a70, k="\006\022tars.tarsquerystat\026\f192.168.8.21&/tars.tarsquerystat.NoTarsObjAdapter.connectRate", 
    v="\t\f", bDirty=bDirty@entry=true, vtData=std::vector of length 0, capacity 0) at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_hashmap.cpp:1885
#7  0x0000000000484161 in PropertyHashMap::add (this=this@entry=0x1fd4a18, head=..., body=...)
    at /usr/local/Tars-1.6.0/framework/TarsFramework/PropertyServer/PropertyHashMap.h:63
Python Exception <type 'exceptions.ValueError'> Cannot find type const std::map<tars::StatPropMsgHead, tars::StatPropMsgBody, std::less<tars::StatPropMsgHead>, std::allocator<std::pair<tars::StatPropMsgHead const, tars::StatPropMsgBody> > >::_Rep_type: 
#8  0x000000000047a5a7 in PropertyImp::handlePropMsg (this=this@entry=0x7fb2440008c0, propMsg=std::map with 5 elements, current=...)
    at /usr/local/Tars-1.6.0/framework/TarsFramework/PropertyServer/PropertyImp.cpp:147
Python Exception <type 'exceptions.ValueError'> Cannot find type const std::map<tars::StatPropMsgHead, tars::StatPropMsgBody, std::less<tars::StatPropMsgHead>, std::allocator<std::pair<tars::StatPropMsgHead const, tars::StatPropMsgBody> > >::_Rep_type: 
#9  0x000000000047aa10 in PropertyImp::reportPropMsg (this=0x7fb2440008c0, propMsg=std::map with 5 elements, current=...)
    at /usr/local/Tars-1.6.0/framework/TarsFramework/PropertyServer/PropertyImp.cpp:82
#10 0x0000000000482be0 in tars::PropertyF::onDispatch (this=0x7fb2440008c0, _current=..., _sResponseBuffer=std::vector of length 0, capacity 0)
    at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/servant/servant/PropertyF.h:620
#11 0x00000000004b6d56 in tars::Servant::dispatch (this=0x7fb2440008c0, current=..., buffer=std::vector of length 0, capacity 0)
    at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/servant/libservant/Servant.cpp:79
#12 0x00000000004bc905 in tars::ServantHandle::handleTarsProtocol (this=0x1fd5cd0, current=...)
    at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/servant/libservant/ServantHandle.cpp:834
#13 0x00000000004bd575 in tars::ServantHandle::handle (this=0x1fd5cd0, stRecvData=...)
    at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/servant/libservant/ServantHandle.cpp:610
#14 0x000000000051ce23 in tars::TC_EpollServer::Handle::handleImp (this=0x1fd5cd0) at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_epoll_server.cpp:216
#15 0x00000000004b7e49 in tars::ServantHandle::run (this=0x1fd5cd0) at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/servant/libservant/ServantHandle.cpp:71
#16 0x000000000053f70a in tars::TC_Thread::threadEntry (pThread=0x1fd5cd0) at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_thread.cpp:93
#17 0x00007fb2d03d5dd5 in start_thread () from /lib64/libpthread.so.0
#18 0x00007fb2cf2bdead in clone () from /lib64/libc.so.6
(gdb) 
#0  tars::TC_MemChunk::allocate (this=0x1fd51a0) at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_mem_chunk.cpp:88
#1  0x00000000005338e1 in allocate2 (iIndex=<synthetic pointer>, this=<optimized out>) at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_mem_chunk.cpp:99
#2  allocate2 (iIndex=<synthetic pointer>, this=<optimized out>) at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_mem_chunk.cpp:246
#3  tars::TC_MemMultiChunkAllocator::allocate2 (this=0x1fd43f0, iNeedSize=162, iAllocSize=@0x7fb2c4ff7418: 162, iIndex=@0x7fb2c4ff7388: 0)
    at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_mem_chunk.cpp:554
#4  0x000000000053397e in tars::TC_MemMultiChunkAllocator::allocate (this=<optimized out>, iNeedSize=<optimized out>, iAllocSize=@0x7fb2c4ff7418: 162)
    at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_mem_chunk.cpp:540
#5  0x000000000052a6c2 in tars::TC_HashMap::BlockAllocator::allocateMemBlock (this=0x1fd43d0, index=index@entry=324881, iAllocSize=@0x7fb2c4ff7418: 162, 
    vtData=std::vector of length 0, capacity 0) at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_hashmap.cpp:857
#6  0x000000000052b2b3 in tars::TC_HashMap::set (this=this@entry=0x1fd4a70, k="\006\022tars.tarsquerystat\026\f192.168.8.21&/tars.tarsquerystat.NoTarsObjAdapter.connectRate", 
    v="\t\f", bDirty=bDirty@entry=true, vtData=std::vector of length 0, capacity 0) at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_hashmap.cpp:1885
#7  0x0000000000484161 in PropertyHashMap::add (this=this@entry=0x1fd4a18, head=..., body=...)
    at /usr/local/Tars-1.6.0/framework/TarsFramework/PropertyServer/PropertyHashMap.h:63
Python Exception <type 'exceptions.ValueError'> Cannot find type const std::map<tars::StatPropMsgHead, tars::StatPropMsgBody, std::less<tars::StatPropMsgHead>, std::allocator<std::pair<tars::StatPropMsgHead const, tars::StatPropMsgBody> > >::_Rep_type: 
#8  0x000000000047a5a7 in PropertyImp::handlePropMsg (this=this@entry=0x7fb2440008c0, propMsg=std::map with 5 elements, current=...)
    at /usr/local/Tars-1.6.0/framework/TarsFramework/PropertyServer/PropertyImp.cpp:147
Python Exception <type 'exceptions.ValueError'> Cannot find type const std::map<tars::StatPropMsgHead, tars::StatPropMsgBody, std::less<tars::StatPropMsgHead>, std::allocator<std::pair<tars::StatPropMsgHead const, tars::StatPropMsgBody> > >::_Rep_type: 
#9  0x000000000047aa10 in PropertyImp::reportPropMsg (this=0x7fb2440008c0, propMsg=std::map with 5 elements, current=...)
    at /usr/local/Tars-1.6.0/framework/TarsFramework/PropertyServer/PropertyImp.cpp:82
#10 0x0000000000482be0 in tars::PropertyF::onDispatch (this=0x7fb2440008c0, _current=..., _sResponseBuffer=std::vector of length 0, capacity 0)
    at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/servant/servant/PropertyF.h:620
#11 0x00000000004b6d56 in tars::Servant::dispatch (this=0x7fb2440008c0, current=..., buffer=std::vector of length 0, capacity 0)
    at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/servant/libservant/Servant.cpp:79
#12 0x00000000004bc905 in tars::ServantHandle::handleTarsProtocol (this=0x1fd5cd0, current=...)
    at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/servant/libservant/ServantHandle.cpp:834
#13 0x00000000004bd575 in tars::ServantHandle::handle (this=0x1fd5cd0, stRecvData=...)
    at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/servant/libservant/ServantHandle.cpp:610
#14 0x000000000051ce23 in tars::TC_EpollServer::Handle::handleImp (this=0x1fd5cd0) at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_epoll_server.cpp:216
#15 0x00000000004b7e49 in tars::ServantHandle::run (this=0x1fd5cd0) at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/servant/libservant/ServantHandle.cpp:71
#16 0x000000000053f70a in tars::TC_Thread::threadEntry (pThread=0x1fd5cd0) at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_thread.cpp:93
#17 0x00007fb2d03d5dd5 in start_thread () from /lib64/libpthread.so.0
#18 0x00007fb2cf2bdead in clone () from /lib64/libc.so.6
(gdb) f 0
#0  tars::TC_MemChunk::allocate (this=0x1fd51a0) at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_mem_chunk.cpp:88
88          _pHead->_firstAvailableBlock = *((size_t *)result);
(gdb) f 1
#1  0x00000000005338e1 in allocate2 (iIndex=<synthetic pointer>, this=<optimized out>) at /usr/local/Tars-1.6.0/framework/TarsFramework/tarscpp/util/src/tc_mem_chunk.cpp:99
99          void *pAddr = allocate();
(gdb)
katechun commented 5 years ago

What language are you using?

php

What operating system (Linux, Ubuntu, …) and version?

linux

What runtime / compiler are you using (e.g. jdk version or version of gcc)

gcc 版本 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC)

Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs). tars所有服务正常启动后，两三分钟后tarsproperty 结果变为 [alarm] down,pid not exist 并且进程ID改变，服务器 /data/tars/app_log/中产生一些 core 文件 image

image

出现同样问题

lvxiaoqi commented 5 years ago

同样问题，求解答

lvxiaoqi commented 5 years ago

@jackylee92 你的php版本是多少？

jackylee92 commented 5 years ago

@lvxiaoqi 7.2.8

copyrenzhe commented 5 years ago

麻烦将 tars-server composer包升级至v0.2.5 试试

jackylee92 commented 5 years ago

@copyrenzhe 我已经升级了！我这边观察下，有线索我再同步给您，谢谢帮助！