search

Vector35 / binaryninja-api

Public API, examples, documentation and issues for Binary Ninja
https://binary.ninja/
MIT License
908 stars 207 forks source link

Name demangler edge cases #1895

Open dfraze opened 4 years ago

dfraze commented 4 years ago

Name demangler seems to have some edge cases it fails on.

binaryninja.demangle.demangle_gnu3(bv.arch, "_ZN14clmdep_msgpack2v123container_size_overflowC1EPKc#2")
(None, '_ZN14clmdep_msgpack2v123container_size_overflowC1EPKc#2')

c++filt says:

$ c++filt
_ZN14clmdep_msgpack2v123container_size_overflowC1EPKc#2
clmdep_msgpack::v1::container_size_overflow::container_size_overflow(char const*)#2

In a private chat, it was suspected that the #2 was the issue. However:

>>> current_function.name
'_ZN11clmdep_asio5error6detail14netdb_categoryD2Ev#2'
>>> binaryninja.demangle.get_qualified_name(binaryninja.demangle.demangle_gnu3(bv.arch, current_function.name)[1])
'clmdep_asio::error::detail::netdb_category::~netdb_category'

A quick one off script reveals other potentially problematic symbol names from the same binary:

demangle_me.txt

plafosse commented 4 years ago

Seems as though most of those demangle fine. There are 50 of them that don't which is obviously a very high percentage.


1 _ZZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtENKUlvE_clEv
2 _ZSt7forwardIZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_EOT_RNSt16remove_referenceISB_E4typeE
3 _ZSt8__invokeIZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_JEENSt15__invoke_resultIT_JDpT0_EE4typeEOSC_DpOSD_
4 _ZNSt6threadC2IZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_JEEEOT_DpOT0_
5 _ZNSt5tupleIJZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_EEC1IJSA_ELb1EEEDpOT_
6 _ZNSt11_Tuple_implILm0EJZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_EEC2ISA_EEOT_
7 _ZNSt10_Head_baseILm0EZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_Lb0EEC2ISA_EEOT_
8 _ZNSt5tupleIJZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_EEC1EOSB_
9 _ZNSt6thread14__make_invokerIZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_JEEENS_8_InvokerISt5tupleIJNSt5decayIT_E4typeEDpNSE_IT0_E4typeEEEEEOSF_DpOSI_
10 _ZSt13__invoke_implIvZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_JEET_St14__invoke_otherOT0_DpOT1_
11 _ZSt7forwardINSt6thread8_InvokerISt5tupleIJZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_EEEEEOT_RNSt16remove_referenceISG_E4typeE
12 _ZNSt6thread13_S_make_stateINS_8_InvokerISt5tupleIJZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_EEEEEESt10unique_ptrINS_6_StateESt14default_deleteISH_EEOT_
13 _ZNSt11_Tuple_implILm0EJZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_EE7_M_headERSB_
14 _ZNSt10_Head_baseILm0EZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_Lb0EE7_M_headERSB_
15 _ZNSt11_Tuple_implILm0EJZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_EEC2EOSB_
16 _ZNSt6thread8_InvokerISt5tupleIJZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_EEEC1EOSE_
17 _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_EEEEEC2EOSF_
18 _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_EEEEED2Ev
19 _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_EEEEED0Ev
20 _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_EEEEE6_M_runEv
21 _ZNSt6thread8_InvokerISt5tupleIJZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_EEEclEv
22 _ZNSt6thread8_InvokerISt5tupleIJZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_EEE9_M_invokeIJLm0EEEEDTcl8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE
23 _ZSt4moveIRSt5tupleIJZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_EEEONSt16remove_referenceIT_E4typeEOSF_
24 _ZSt3getILm0EJZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_EERNSt13tuple_elementIXT_ESt5tupleIJDpT0_EEE4typeERSF_
25 _ZSt12__get_helperILm0EZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_JEERT0_RSt11_Tuple_implIXT_EJSB_DpT1_EE
26 _ZSt3getILm0EJZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_EEONSt13tuple_elementIXT_ESt5tupleIJDpT0_EEE4typeEOSF_
27 _ZSt7forwardIOZN3rpc6clientC4ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEtEUlvE_EOT_RNSt16remove_referenceISC_E4typeE
28 _ZNSt6chrono13duration_castINS_8durationIlSt5ratioILl1ELl1EEEElS2_ILl1ELl1000000000EEEENSt9enable_ifIXsrNS_13__is_durationIT_EE5valueES8_E4typeERKNS1_IT0_T1_EE
29 _ZNSt6chrono13duration_castINS_8durationIlSt5ratioILl1ELl1000000000EEEElS2_ILl1ELl1EEEENSt9enable_ifIXsrNS_13__is_durationIT_EE5valueES8_E4typeERKNS1_IT0_T1_EE
30 _ZSt4swapINSt6thread2idEENSt9enable_ifIXsrSt6__and_IJSt6__not_ISt15__is_tuple_likeIT_EESt21is_move_constructibleIS6_ESt18is_move_assignableIS6_EEE5valueEvE4typeERS6_SG_
31 _ZZSt9call_onceIMNSt13__future_base13_State_baseV2EFvPSt8functionIFSt10unique_ptrINS0_12_Result_baseENS4_8_DeleterEEvEEPbEJPS1_S9_SA_EEvRSt9once_flagOT_DpOT0_ENKUlvE_clEv
32 _ZZSt9call_onceIMNSt13__future_base13_State_baseV2EFvPSt8functionIFSt10unique_ptrINS0_12_Result_baseENS4_8_DeleterEEvEEPbEJPS1_S9_SA_EEvRSt9once_flagOT_DpOT0_ENKUlvE0_clEv
33 _ZZSt9call_onceIMNSt13__future_base13_State_baseV2EFvPSt8functionIFSt10unique_ptrINS0_12_Result_baseENS4_8_DeleterEEvEEPbEJPS1_S9_SA_EEvRSt9once_flagOT_DpOT0_ENUlvE0_4_FUNEv
34 _ZZSt9call_onceIMNSt13__future_base13_State_baseV2EFvPSt8functionIFSt10unique_ptrINS0_12_Result_baseENS4_8_DeleterEEvEEPbEJPS1_S9_SA_EEvRSt9once_flagOT_DpOT0_ENKUlvE0_cvPFvvEEv
35 _ZNSt10error_codeaSIN11clmdep_asio5error12basic_errorsEEENSt9enable_ifIXsrSt18is_error_code_enumIT_E5valueERS_E4typeES6_
36 _ZNSt10error_codeaSIN11clmdep_asio5error11misc_errorsEEENSt9enable_ifIXsrSt18is_error_code_enumIT_E5valueERS_E4typeES6_
37 _ZNSt12__shared_ptrIvLN9__gnu_cxx12_Lock_policyE2EE5resetIvN11clmdep_asio6detail10socket_ops12noop_deleterEEENSt9enable_ifIXsrSt21__sp_is_constructibleIvT_E5valueEvE4typeEPSA_T0_
38 _ZNSt6chrono13duration_castINS_8durationIlSt5ratioILl1ELl1000000000EEEElS2_ILl1ELl1000EEEENSt9enable_ifIXsrNS_13__is_durationIT_EE5valueES8_E4typeERKNS1_IT0_T1_EE
39 _ZNSt13unordered_mapIjSt4pairINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt7promiseIN14clmdep_msgpack2v113object_handleEEESt4hashIjESt8equal_toIjESaIS0_IKjSC_EEE6insertIS0_IiSC_EEENSt9enable_ifIXsrSt16is_constructibleISI_JOT_EE5valueES0_INSt8__detail14_Node_iteratorISI_Lb0ELb0EEEbEE4typeESQ_
40 _ZSt11__addressofIZSt9call_onceIMNSt13__future_base13_State_baseV2EFvPSt8functionIFSt10unique_ptrINS1_12_Result_baseENS5_8_DeleterEEvEEPbEJPS2_SA_SB_EEvRSt9once_flagOT_DpOT0_EUlvE_EPSH_RSH_
41 _ZSt4swapISt15__uniq_ptr_implINSt13__future_base12_Result_baseENS2_8_DeleterEEENSt9enable_ifIXsrSt6__and_IJSt6__not_ISt15__is_tuple_likeIT_EESt21is_move_constructibleIS9_ESt18is_move_assignableIS9_EEE5valueEvE4typeERS9_SJ_
42 _ZSt4swapIPN14clmdep_msgpack2v14zoneEENSt9enable_ifIXsrSt6__and_IJSt6__not_ISt15__is_tuple_likeIT_EESt21is_move_constructibleIS8_ESt18is_move_assignableIS8_EEE5valueEvE4typeERS8_SI_
43 _ZSt4swapIPvENSt9enable_ifIXsrSt6__and_IJSt6__not_ISt15__is_tuple_likeIT_EESt21is_move_constructibleIS5_ESt18is_move_assignableIS5_EEE5valueEvE4typeERS5_SF_
44 _ZNSt6chrono13duration_castINS_8durationIlSt5ratioILl1ELl1000000000EEEElS3_EENSt9enable_ifIXsrNS_13__is_durationIT_EE5valueES7_E4typeERKNS1_IT0_T1_EE
45 _ZNSt12__shared_ptrISt6vectorIN11clmdep_asio2ip20basic_resolver_entryINS2_3tcpEEESaIS5_EELN9__gnu_cxx12_Lock_policyE2EE5resetIS7_EENSt9enable_ifIXsrSt21__sp_is_constructibleIS7_T_E5valueEvE4typeEPSE_
46 _ZNSt23_Sp_counted_ptr_inplaceINSt13__future_base13_State_baseV2ESaIS1_ELN9__gnu_cxx12_Lock_policyE2EEC2IJEEES2_DpOT_
47 _ZSt12__miter_baseIPPN14clmdep_msgpack2v26objectEEDTcl12__miter_basecldtfp_4baseEEESt13move_iteratorIT_E
48 _ZSt10__fill_n_aIPPN14clmdep_msgpack2v26objectEmS3_EN9__gnu_cxx11__enable_ifIXsrSt11__is_scalarIT1_E7__valueET_E6__typeESA_T0_RKS8_
49 _ZSt4swapIPSt6vectorIN11clmdep_asio2ip20basic_resolver_entryINS2_3tcpEEESaIS5_EEENSt9enable_ifIXsrSt6__and_IJSt6__not_ISt15__is_tuple_likeIT_EESt21is_move_constructibleISD_ESt18is_move_assignableISD_EEE5valueEvE4typeERSD_SN_
50 _ZNSt4pairIKjS_INSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt7promiseIN14clmdep_msgpack2v113object_handleEEEEC2IJRS0_EJLm0EEJEJEEERSt5tupleIJDpT_EERSG_IJDpT1_EESt12_Index_tupleIJXspT0_EEESP_IJXspT2_EEE```
dfraze commented 4 years ago

I may have pasted the before list instead of the after list. My mistake.

plafosse commented 4 years ago

Hmm that symbol with the #2 at the doesn't look like its a part of the original file you posted. Any idea on that?

plafosse commented 4 years ago

As of build 2.1.2347 only the following 18 fail to demangle:


_ZZSt9call_onceIMNSt13__future_base13_State_baseV2EFvPSt8functionIFSt10unique_ptrINS0_12_Result_baseENS4_8_DeleterEEvEEPbEJPS1_S9_SA_EEvRSt9once_flagOT_DpOT0_ENKUlvE_clEv
_ZZSt9call_onceIMNSt13__future_base13_State_baseV2EFvPSt8functionIFSt10unique_ptrINS0_12_Result_baseENS4_8_DeleterEEvEEPbEJPS1_S9_SA_EEvRSt9once_flagOT_DpOT0_ENKUlvE0_clEv
_ZZSt9call_onceIMNSt13__future_base13_State_baseV2EFvPSt8functionIFSt10unique_ptrINS0_12_Result_baseENS4_8_DeleterEEvEEPbEJPS1_S9_SA_EEvRSt9once_flagOT_DpOT0_ENUlvE0_4_FUNEv
_ZZSt9call_onceIMNSt13__future_base13_State_baseV2EFvPSt8functionIFSt10unique_ptrINS0_12_Result_baseENS4_8_DeleterEEvEEPbEJPS1_S9_SA_EEvRSt9once_flagOT_DpOT0_ENKUlvE0_cvPFvvEEv
_ZNSt10error_codeaSIN11clmdep_asio5error12basic_errorsEEENSt9enable_ifIXsrSt18is_error_code_enumIT_E5valueERS_E4typeES6_
_ZNSt10error_codeaSIN11clmdep_asio5error11misc_errorsEEENSt9enable_ifIXsrSt18is_error_code_enumIT_E5valueERS_E4typeES6_
_ZNSt12__shared_ptrIvLN9__gnu_cxx12_Lock_policyE2EE5resetIvN11clmdep_asio6detail10socket_ops12noop_deleterEEENSt9enable_ifIXsrSt21__sp_is_constructibleIvT_E5valueEvE4typeEPSA_T0_
_ZNSt13unordered_mapIjSt4pairINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt7promiseIN14clmdep_msgpack2v113object_handleEEESt4hashIjESt8equal_toIjESaIS0_IKjSC_EEE6insertIS0_IiSC_EEENSt9enable_ifIXsrSt16is_constructibleISI_JOT_EE5valueES0_INSt8__detail14_Node_iteratorISI_Lb0ELb0EEEbEE4typeESQ_
_ZSt11__addressofIZSt9call_onceIMNSt13__future_base13_State_baseV2EFvPSt8functionIFSt10unique_ptrINS1_12_Result_baseENS5_8_DeleterEEvEEPbEJPS2_SA_SB_EEvRSt9once_flagOT_DpOT0_EUlvE_EPSH_RSH_
_ZSt4swapISt15__uniq_ptr_implINSt13__future_base12_Result_baseENS2_8_DeleterEEENSt9enable_ifIXsrSt6__and_IJSt6__not_ISt15__is_tuple_likeIT_EESt21is_move_constructibleIS9_ESt18is_move_assignableIS9_EEE5valueEvE4typeERS9_SJ_
_ZSt4swapIPN14clmdep_msgpack2v14zoneEENSt9enable_ifIXsrSt6__and_IJSt6__not_ISt15__is_tuple_likeIT_EESt21is_move_constructibleIS8_ESt18is_move_assignableIS8_EEE5valueEvE4typeERS8_SI_
_ZSt4swapIPvENSt9enable_ifIXsrSt6__and_IJSt6__not_ISt15__is_tuple_likeIT_EESt21is_move_constructibleIS5_ESt18is_move_assignableIS5_EEE5valueEvE4typeERS5_SF_
_ZNSt12__shared_ptrISt6vectorIN11clmdep_asio2ip20basic_resolver_entryINS2_3tcpEEESaIS5_EELN9__gnu_cxx12_Lock_policyE2EE5resetIS7_EENSt9enable_ifIXsrSt21__sp_is_constructibleIS7_T_E5valueEvE4typeEPSE_
_ZNSt23_Sp_counted_ptr_inplaceINSt13__future_base13_State_baseV2ESaIS1_ELN9__gnu_cxx12_Lock_policyE2EEC2IJEEES2_DpOT_
_ZSt10__fill_n_aIPPN14clmdep_msgpack2v26objectEmS3_EN9__gnu_cxx11__enable_ifIXsrSt11__is_scalarIT1_E7__valueET_E6__typeESA_T0_RKS8_
_ZSt4swapIPSt6vectorIN11clmdep_asio2ip20basic_resolver_entryINS2_3tcpEEESaIS5_EEENSt9enable_ifIXsrSt6__and_IJSt6__not_ISt15__is_tuple_likeIT_EESt21is_move_constructibleISD_ESt18is_move_assignableISD_EEE5valueEvE4typeERSD_SN_
_ZNSt4pairIKjS_INSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt7promiseIN14clmdep_msgpack2v113object_handleEEEEC2IJRS0_EJLm0EEJEJEEERSt5tupleIJDpT_EERSG_IJDpT1_EESt12_Index_tupleIJXspT0_EEESP_IJXspT2_EEE
dfraze commented 4 years ago

Hmm that symbol with the #2 at the doesn't look like its a part of the original file you posted. Any idea on that?

I can't find it now.

yrp604 commented 3 years ago

A few from vmwp.exe that failed to demangle:

??1?$VmMethodDelegate@VVirtualMachine@@XW4VmGuestStopType@@@Vml@@UEAA@XZ
??1?$unique_struct@URESTORABLE_CONTEXT@_VML_ETW_TRACE@@P6AXPEAU12@@Z$1?RestoreThreadLocalContext@2@SAX0@Z$$T$0A@@wil@@QEAA@XZ
??1?$unique_any_t@V?$event_t@V?$unique_storage@U?$resource_policy@PEAXP6AXPEAX@Z$1?CloseHandle@details@wil@@YAX0@ZU?$integral_constant@_K$0A@@wistd@@PEAXPEAX$0A@$$T@details@wil@@@details@wil@@Uerr_exception_policy@3@@wil@@@wil@@QEAA@XZ
CouleeApps commented 3 years ago

Some from /usr/lib/dyld:

____ZN4dyld5_mainEPK12macho_headermiPPKcS5_S5_Pm_block_invoke.110
____ZN4dyld5_mainEPK12macho_headermiPPKcS5_S5_Pm_block_invoke_2
____ZN5dyld37closure13PathOverrides17setMainExecutableEPKNS_9MachOFileEPKc_block_invoke

I think these are objective-c related

fuzyll commented 2 years ago

__ZTIN5realm24CollectionChangeCallback4ImplIZ23RLMAddNotificationBlockINS_4ListEEP20RLMNotificationTokenP11objc_objectRT_U13block_pointerFvS7_P19RLMCollectionChangeP7NSErrorEbEUlRKNS_19CollectionChangeSetESt13exception_ptrE_EE demangles with c++filt, but not demangle_gnu3

psifertex commented 2 years ago

Related issue: #1653

CouleeApps commented 6 months ago

Here's another 3000 cases from various binaries (looks like mostly webkit):

demanglegnu_missing.txt