search

williballenthin / siglib

function identification signatures
Apache License 2.0
11 stars 1 forks source link

Generate major .sig files #21

Closed mr-tz closed 3 years ago

mr-tz commented 3 years ago
mr-tz commented 3 years ago

32-bit run-time functions

created using script proposed in #27 

$ python3 create_sig.py -d -e libraries amd64 -ep x64 amd64 -ip libc msvc libvc vcruntime libucrt legacy_ oldnames libcon --tarballs-root data/ --no-act -- flare_rtf_vc32/ 2>&1 | grep -v "\[-\]"
DEBUG:__main__:extracting from data/VS10\VS10.tar.gz
DEBUG:__main__: [+] VS10/VC/lib/libcmt.lib.pat
DEBUG:__main__: [+] VS10/VC/lib/libcmtd.lib.pat
DEBUG:__main__: [+] VS10/VC/lib/libcpmt.lib.pat
DEBUG:__main__: [+] VS10/VC/lib/libcpmt1.lib.pat
DEBUG:__main__: [+] VS10/VC/lib/libcpmtd.lib.pat
DEBUG:__main__: [+] VS10/VC/lib/libcpmtd0.lib.pat
DEBUG:__main__: [+] VS10/VC/lib/libcpmtd1.lib.pat
DEBUG:__main__: [+] VS10/VC/lib/msvcmrt.lib.pat
DEBUG:__main__: [+] VS10/VC/lib/msvcmrtd.lib.pat
DEBUG:__main__: [+] VS10/VC/lib/msvcprt.lib.pat
DEBUG:__main__: [+] VS10/VC/lib/msvcprtd.lib.pat
DEBUG:__main__: [+] VS10/VC/lib/msvcrt.lib.pat
DEBUG:__main__: [+] VS10/VC/lib/msvcrtd.lib.pat
DEBUG:__main__: [+] VS10/VC/lib/msvcurt.lib.pat
DEBUG:__main__: [+] VS10/VC/lib/msvcurtd.lib.pat
DEBUG:__main__:extracting from data/VS11\VS11.tar.gz
DEBUG:__main__: [+] VS11/VC/lib/libcmt.lib.pat
DEBUG:__main__: [+] VS11/VC/lib/libcmtd.lib.pat
DEBUG:__main__: [+] VS11/VC/lib/libcpmt.lib.pat
DEBUG:__main__: [+] VS11/VC/lib/libcpmt1.lib.pat
DEBUG:__main__: [+] VS11/VC/lib/libcpmtd.lib.pat
DEBUG:__main__: [+] VS11/VC/lib/libcpmtd0.lib.pat
DEBUG:__main__: [+] VS11/VC/lib/libcpmtd1.lib.pat
DEBUG:__main__: [+] VS11/VC/lib/msvcmrt.lib.pat
DEBUG:__main__: [+] VS11/VC/lib/msvcmrtd.lib.pat
DEBUG:__main__: [+] VS11/VC/lib/msvcprt.lib.pat
DEBUG:__main__: [+] VS11/VC/lib/msvcprtd.lib.pat
DEBUG:__main__: [+] VS11/VC/lib/msvcrt.lib.pat
DEBUG:__main__: [+] VS11/VC/lib/msvcrtd.lib.pat
DEBUG:__main__: [+] VS11/VC/lib/msvcurt.lib.pat
DEBUG:__main__: [+] VS11/VC/lib/msvcurtd.lib.pat
DEBUG:__main__:extracting from data/VS12\VS12.tar.gz
DEBUG:__main__: [+] VS12/VC/lib/libcmt.lib.pat
DEBUG:__main__: [+] VS12/VC/lib/libcmtd.lib.pat
DEBUG:__main__: [+] VS12/VC/lib/libcpmt.lib.pat
DEBUG:__main__: [+] VS12/VC/lib/libcpmt1.lib.pat
DEBUG:__main__: [+] VS12/VC/lib/libcpmtd.lib.pat
DEBUG:__main__: [+] VS12/VC/lib/libcpmtd0.lib.pat
DEBUG:__main__: [+] VS12/VC/lib/libcpmtd1.lib.pat
DEBUG:__main__: [+] VS12/VC/lib/msvcmrt.lib.pat
DEBUG:__main__: [+] VS12/VC/lib/msvcmrtd.lib.pat
DEBUG:__main__: [+] VS12/VC/lib/msvcprt.lib.pat
DEBUG:__main__: [+] VS12/VC/lib/msvcprtd.lib.pat
DEBUG:__main__: [+] VS12/VC/lib/msvcrt.lib.pat
DEBUG:__main__: [+] VS12/VC/lib/msvcrtd.lib.pat
DEBUG:__main__: [+] VS12/VC/lib/msvcurt.lib.pat
DEBUG:__main__: [+] VS12/VC/lib/msvcurtd.lib.pat
DEBUG:__main__:extracting from data/VS2015\compiler\14.0.tar.gz
DEBUG:__main__: [+] 14.0/lib/vcruntimed.lib.pat
DEBUG:__main__: [+] 14.0/lib/vcruntime.lib.pat
DEBUG:__main__: [+] 14.0/lib/oldnames.lib.pat
DEBUG:__main__: [+] 14.0/lib/msvcurtd.lib.pat
DEBUG:__main__: [+] 14.0/lib/msvcurt.lib.pat
DEBUG:__main__: [+] 14.0/lib/msvcrtd.lib.pat
DEBUG:__main__: [+] 14.0/lib/msvcrt.lib.pat
DEBUG:__main__: [+] 14.0/lib/msvcprtd.lib.pat
DEBUG:__main__: [+] 14.0/lib/msvcprt.lib.pat
DEBUG:__main__: [+] 14.0/lib/msvcmrtd.lib.pat
DEBUG:__main__: [+] 14.0/lib/msvcmrt.lib.pat
DEBUG:__main__: [+] 14.0/lib/libvcruntimed.lib.pat
DEBUG:__main__: [+] 14.0/lib/libvcruntimed.lib.pat
DEBUG:__main__: [+] 14.0/lib/libvcruntime.lib.pat
DEBUG:__main__: [+] 14.0/lib/libvcruntime.lib.pat
DEBUG:__main__: [+] 14.0/lib/libcpmtd1.lib.pat
DEBUG:__main__: [+] 14.0/lib/libcpmtd0.lib.pat
DEBUG:__main__: [+] 14.0/lib/libcpmtd.lib.pat
DEBUG:__main__: [+] 14.0/lib/libcpmt1.lib.pat
DEBUG:__main__: [+] 14.0/lib/libcpmt.lib.pat
DEBUG:__main__: [+] 14.0/lib/libconcrtd1.lib.pat
DEBUG:__main__: [+] 14.0/lib/libconcrtd1.lib.pat
DEBUG:__main__: [+] 14.0/lib/libconcrtd0.lib.pat
DEBUG:__main__: [+] 14.0/lib/libconcrtd0.lib.pat
DEBUG:__main__: [+] 14.0/lib/libconcrtd.lib.pat
DEBUG:__main__: [+] 14.0/lib/libconcrtd.lib.pat
DEBUG:__main__: [+] 14.0/lib/libconcrt1.lib.pat
DEBUG:__main__: [+] 14.0/lib/libconcrt1.lib.pat
DEBUG:__main__: [+] 14.0/lib/libconcrt.lib.pat
DEBUG:__main__: [+] 14.0/lib/libconcrt.lib.pat
DEBUG:__main__: [+] 14.0/lib/libcmtd.lib.pat
DEBUG:__main__: [+] 14.0/lib/libcmt.lib.pat
DEBUG:__main__: [+] 14.0/lib/legacy_stdio_wide_specifiers.lib.pat
DEBUG:__main__: [+] 14.0/lib/legacy_stdio_definitions.lib.pat
DEBUG:__main__:extracting from data/VS2015\ucrt\10.0.10240.0.tar.gz
DEBUG:__main__: [+] 10.0.10240.0/ucrt/x86/libucrt.lib.pat
DEBUG:__main__: [+] 10.0.10240.0/ucrt/x86/libucrtd.lib.pat
DEBUG:__main__:extracting from data/VS2017\compiler\14.16.27023.tar.gz
DEBUG:__main__: [+] 14.16.27023/lib/x86/vcruntimed.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/vcruntime.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/oldnames.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/msvcurtd.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/msvcrtd.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/msvcrt.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/msvcprtd.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/msvcprt.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/msvcmrtd.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/msvcmrt.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/msvcurt.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libvcruntime.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libvcruntime.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libvcruntimed.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libvcruntimed.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libcpmtd1.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libcpmtd0.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libcpmtd.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libcpmt1.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libcpmt.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libconcrtd1.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libconcrtd1.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libconcrtd0.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libconcrtd0.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libconcrtd.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libconcrtd.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libconcrt1.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libconcrt1.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libconcrt.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libconcrt.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libcmtd.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/libcmt.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/legacy_stdio_wide_specifiers.lib.pat
DEBUG:__main__: [+] 14.16.27023/lib/x86/legacy_stdio_definitions.lib.pat
DEBUG:__main__:extracting from data/VS2017\ucrt\10.0.16299.0.tar.gz
DEBUG:__main__: [+] 10.0.16299.0/ucrt/x86/libucrtd.lib.pat
DEBUG:__main__: [+] 10.0.16299.0/ucrt/x86/libucrt.lib.pat
DEBUG:__main__: [+] 10.0.16299.0/um/x86/ntstc_libcmt.lib.pat
DEBUG:__main__: [+] 10.0.16299.0/um/x86/ntstc_msvcrt.lib.pat
DEBUG:__main__: [+] 10.0.16299.0/um/x86/wsmsvc.lib.pat
DEBUG:__main__:extracting from data/VS2017\ucrt\10.0.17134.0.tar.gz
DEBUG:__main__: [+] 10.0.17134.0/ucrt/x86/libucrtd.lib.pat
DEBUG:__main__: [+] 10.0.17134.0/ucrt/x86/libucrt.lib.pat
DEBUG:__main__: [+] 10.0.17134.0/um/x86/ntstc_libcmt.lib.pat
DEBUG:__main__: [+] 10.0.17134.0/um/x86/ntstc_msvcrt.lib.pat
DEBUG:__main__: [+] 10.0.17134.0/um/x86/wsmsvc.lib.pat
DEBUG:__main__:extracting from data/VS2017\ucrt\10.0.17763.0.tar.gz
DEBUG:__main__: [+] 10.0.17763.0/um/x86/wsmsvc.lib.pat
DEBUG:__main__: [+] 10.0.17763.0/um/x86/ntstc_msvcrt.lib.pat
DEBUG:__main__: [+] 10.0.17763.0/um/x86/ntstc_libcmt.lib.pat
DEBUG:__main__: [+] 10.0.17763.0/ucrt/x86/libucrtd.lib.pat
DEBUG:__main__: [+] 10.0.17763.0/ucrt/x86/libucrt.lib.pat
DEBUG:__main__:extracting from data/VS2019\compiler\14.28.29910.tar.gz
DEBUG:__main__: [+] 14.28.29910/lib/x86/vcruntimed.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/vcruntime.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/plegacy_stdio_float_rounding.obj.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/oldnames.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/msvcurt_netcore.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/msvcurtd_netcore.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/msvcurtd.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/msvcurt.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/msvcrtd.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/msvcrt.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/msvcprtd.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/msvcprt.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/msvcmrt_netcore.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/msvcmrtd_netcore.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/msvcmrtd.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/msvcmrt.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libvcruntimed.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libvcruntimed.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libvcruntime.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libvcruntime.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libvcasand.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libvcasan.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libcpmtd1.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libcpmtd0.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libcpmtd.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libcpmt1.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libcpmt.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libconcrtd1.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libconcrtd1.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libconcrtd0.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libconcrtd0.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libconcrtd.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libconcrtd.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libconcrt1.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libconcrt1.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libconcrt.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libconcrt.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libcmtd.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/libcmt.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/legacy_x86_flt_exceptions.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/legacy_stdio_wide_specifiers.lib.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/legacy_stdio_float_rounding.obj.pat
DEBUG:__main__: [+] 14.28.29910/lib/x86/legacy_stdio_definitions.lib.pat
DEBUG:__main__:extracting from data/VS2019\ucrt\10.0.18362.0.tar.gz
DEBUG:__main__: [+] 10.0.18362.0/ucrt/x86/libucrtd.lib.pat
DEBUG:__main__: [+] 10.0.18362.0/ucrt/x86/libucrt.lib.pat
DEBUG:__main__: [+] 10.0.18362.0/um/x86/wsmsvc.lib.pat
DEBUG:__main__: [+] 10.0.18362.0/um/x86/ntstc_msvcrt.lib.pat
DEBUG:__main__: [+] 10.0.18362.0/um/x86/ntstc_libcmt.lib.pat
DEBUG:__main__:extracting from data/VS2019\ucrt\10.0.19041.0.tar.gz
DEBUG:__main__: [+] 10.0.19041.0/um/x86/wsmsvc.lib.pat
DEBUG:__main__: [+] 10.0.19041.0/um/x86/ntstc_msvcrt.lib.pat
DEBUG:__main__: [+] 10.0.19041.0/um/x86/ntstc_libcmt.lib.pat
DEBUG:__main__: [+] 10.0.19041.0/ucrt/x86/libucrtd.lib.pat
DEBUG:__main__: [+] 10.0.19041.0/ucrt/x86/libucrt.lib.pat
DEBUG:__main__:extracting from data/VS6\VS6.tar.gz
DEBUG:__main__: [+] VS6/vc98/lib/libc.lib.pat
DEBUG:__main__: [+] VS6/vc98/lib/libcd.lib.pat
DEBUG:__main__: [+] VS6/vc98/lib/libcimt.lib.pat
DEBUG:__main__: [+] VS6/vc98/lib/libcimtd.lib.pat
DEBUG:__main__: [+] VS6/vc98/lib/libcmt.lib.pat
DEBUG:__main__: [+] VS6/vc98/lib/libcmtd.lib.pat
DEBUG:__main__: [+] VS6/vc98/lib/libcp.lib.pat
DEBUG:__main__: [+] VS6/vc98/lib/libcpd.lib.pat
DEBUG:__main__: [+] VS6/vc98/lib/libcpmt.lib.pat
DEBUG:__main__: [+] VS6/vc98/lib/libcpmtd.lib.pat
DEBUG:__main__: [+] VS6/vc98/lib/msvcprt.lib.pat
DEBUG:__main__: [+] VS6/vc98/lib/msvcprtd.lib.pat
DEBUG:__main__: [+] VS6/vc98/lib/msvcrt.lib.pat
DEBUG:__main__: [+] VS6/vc98/lib/msvcrtd.lib.pat
DEBUG:__main__:extracting from data/VS8\VS8.tar.gz
DEBUG:__main__: [+] VS8/VC/lib/libcmt.lib.pat
DEBUG:__main__: [+] VS8/VC/lib/libcmtd.lib.pat
DEBUG:__main__: [+] VS8/VC/lib/libcpmt.lib.pat
DEBUG:__main__: [+] VS8/VC/lib/libcpmtd.lib.pat
DEBUG:__main__: [+] VS8/VC/lib/msvcmrt.lib.pat
DEBUG:__main__: [+] VS8/VC/lib/msvcmrtd.lib.pat
DEBUG:__main__: [+] VS8/VC/lib/msvcprt.lib.pat
DEBUG:__main__: [+] VS8/VC/lib/msvcprtd.lib.pat
DEBUG:__main__: [+] VS8/VC/lib/msvcrt.lib.pat
DEBUG:__main__: [+] VS8/VC/lib/msvcrtd.lib.pat
DEBUG:__main__: [+] VS8/VC/lib/msvcurt.lib.pat
DEBUG:__main__: [+] VS8/VC/lib/msvcurtd.lib.pat
DEBUG:__main__:extracting from data/VS9\VS9.tar.gz
DEBUG:__main__: [+] VS9/VC/lib/libcmt.lib.pat
DEBUG:__main__: [+] VS9/VC/lib/libcmtd.lib.pat
DEBUG:__main__: [+] VS9/VC/lib/libcpmt.lib.pat
DEBUG:__main__: [+] VS9/VC/lib/libcpmtd.lib.pat
DEBUG:__main__: [+] VS9/VC/lib/msvcmrt.lib.pat
DEBUG:__main__: [+] VS9/VC/lib/msvcmrtd.lib.pat
DEBUG:__main__: [+] VS9/VC/lib/msvcprt.lib.pat
DEBUG:__main__: [+] VS9/VC/lib/msvcprtd.lib.pat
DEBUG:__main__: [+] VS9/VC/lib/msvcrt.lib.pat
DEBUG:__main__: [+] VS9/VC/lib/msvcrtd.lib.pat
DEBUG:__main__: [+] VS9/VC/lib/msvcurt.lib.pat
DEBUG:__main__: [+] VS9/VC/lib/msvcurtd.lib.pat
mr-tz commented 3 years ago

Added first set in https://github.com/williballenthin/siglib/commit/ce16b75c7a9f27a7dcf07d9adcee009870d1a3ff

mr-tz commented 3 years ago

Rough stats compared to IDA sigs running on capa-testfiles using match_flirt

RTF

functions percentage
same IDA vs. FLARE 72%
recognized as lib by FLARE but no name, name in IDA 16%
not recognized by FLARE, recognized by IDA 12%
not recognized by IDA, recognized by FLARE 8%

ATL/MFC

functions percentage
same IDA vs. FLARE 82%
recognized as lib by FLARE but no name, name in IDA 5%
not recognized by FLARE, recognized by IDA 6%
not recognized by IDA, recognized by FLARE 13%

Common Libraries

Recognizes 9208 functions in 174 capa-testfiles