search

mandiant / flare-floss

FLARE Obfuscated String Solver - Automatically extract obfuscated strings from malware.
Apache License 2.0
3.12k stars 448 forks source link

rendering slow (language strings) #870

Open mr-tz opened 10 months ago

mr-tz commented 10 months ago

some examples timing floss runs on my system, analysis takes a few seconds, rest of the work is spend rendering (in verbose -v mode):

072e3cc7477f29287f0625b8f22d2d82a66407553859ec99c719087a680125ab
INFO: floss.language.identify: Go binary found with version 1.18
INFO: floss: extracting language-specific Go strings
INFO: floss: finished execution after 8.08 seconds
INFO: floss: rendering results
0.06s user 1.75s system 6% cpu 25.983 total

116bef3f2d4d739bc890726413bd4906a6a673a59128bc9059256b7ebe5888c0
INFO: floss.language.identify: Go binary found with version 1.18
INFO: floss: extracting language-specific Go strings
INFO: floss: finished execution after 3.07 seconds
INFO: floss: rendering results
0.04s user 0.86s system 7% cpu 12.311 total

12bd5b19b7a74d7de015862a8bbc8a6496c071a243461853ed78e4dbd1142fe7
INFO: floss.language.identify: Go binary found with version 1.18
INFO: floss: extracting language-specific Go strings
INFO: floss: finished execution after 101.12 seconds
INFO: floss: rendering results
0.25s user 6.54s system 4% cpu 2:45.87 total

19d9d116e8c116882d618dd78fde8b5798eaa1e6b86a6007502cfc4d32e1bd2d
INFO: floss.language.identify: Go binary found with version 1.18
INFO: floss: extracting language-specific Go strings
INFO: floss: finished execution after 8.24 seconds
INFO: floss: rendering results
0.07s user 1.97s system 7% cpu 28.570 total

1e22b24e5b80926ede6c28d4f1eeb6252ce9f26f99e320d06ae012e489ebe40f
INFO: floss: finished execution after 2.11 seconds
INFO: floss: rendering results
0.15s user 4.64s system 9% cpu 48.345 total

1fdd1c057c3c31044400ef6ade20ad3f10bce415ad33ccfb4bc2fd83bb36f62f
INFO: floss: finished execution after 8.47 seconds
INFO: floss: rendering results
0.46s user 16.35s system 10% cpu 2:41.52 total

2258e78d74da6296a9d5863246f47697e8a3ddf461d55ab7a7eb91702f73cf30
INFO: floss.language.identify: Go binary found with version 1.20
INFO: floss: extracting language-specific Go strings
INFO: floss: finished execution after 3.18 seconds
INFO: floss: rendering results
0.01s user 0.89s system 7% cpu 12.819 total
williballenthin commented 10 months ago

if the slow code isn't obvious, i've had good luck using py-spy for profiling.

mr-tz commented 8 months ago

already a little bit better now:

072e3cc7477f29287f0625b8f22d2d82a66407553859ec99c719087a680125ab
INFO: floss.language.identify: Go binary found with version 1.18
INFO: floss: extracting language-specific Go strings
INFO: floss: finished execution after 0.98 seconds
INFO: floss: rendering results
floss.exe   0.03s user 1.30s system 10% cpu 13.190 total
116bef3f2d4d739bc890726413bd4906a6a673a59128bc9059256b7ebe5888c0
INFO: floss.language.identify: Go binary found with version 1.18
INFO: floss: extracting language-specific Go strings
INFO: floss: finished execution after 0.86 seconds
INFO: floss: rendering results
floss.exe   0.03s user 0.58s system 8% cpu 7.019 total
12bd5b19b7a74d7de015862a8bbc8a6496c071a243461853ed78e4dbd1142fe7
INFO: floss.language.identify: Go binary found with version 1.18
INFO: floss: extracting language-specific Go strings
INFO: floss: finished execution after 5.59 seconds
INFO: floss: rendering results
floss.exe   0.11s user 4.66s system 9% cpu 48.797 total
19d9d116e8c116882d618dd78fde8b5798eaa1e6b86a6007502cfc4d32e1bd2d
INFO: floss.language.identify: Go binary found with version 1.18
INFO: floss: extracting language-specific Go strings
INFO: floss: finished execution after 0.70 seconds
INFO: floss: rendering results
floss.exe   0.05s user 1.38s system 10% cpu 13.728 total
1e22b24e5b80926ede6c28d4f1eeb6252ce9f26f99e320d06ae012e489ebe40f
INFO: floss: finished execution after 1.51 seconds
INFO: floss: rendering results
floss.exe   0.07s user 3.36s system 10% cpu 31.555 total
1fdd1c057c3c31044400ef6ade20ad3f10bce415ad33ccfb4bc2fd83bb36f62f
INFO: floss: finished execution after 3.90 seconds
INFO: floss: rendering results
floss.exe   0.33s user 10.67s system 10% cpu 1:42.25 total
2258e78d74da6296a9d5863246f47697e8a3ddf461d55ab7a7eb91702f73cf30
INFO: floss.language.identify: Go binary found with version 1.20
INFO: floss: extracting language-specific Go strings
INFO: floss: finished execution after 0.47 seconds
INFO: floss: rendering results
floss.exe   0.02s user 0.57s system 9% cpu 6.183 total
mr-tz commented 8 months ago

details for 12bd5b19b7a74d7de015862a8bbc8a6496c071a243461853ed78e4dbd1142fe7:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    814/1    0.014    0.000   31.716   31.716 {built-in method builtins.exec}
        1    0.007    0.007   31.716   31.716 floss/main.py:1(<module>)
        1    0.003    0.003   30.819   30.819 floss/main.py:458(main)
        1    0.001    0.001   21.708   21.708 flare-floss\floss\render\default.py:302(render)
    72213    0.443    0.000   20.344    0.000 flare-floss\venv\lib\site-packages\rich\console.py:1623(print)
        1    0.017    0.017   19.165   19.165 flare-floss\floss\render\default.py:170(render_staticstrings)
        2    0.305    0.152   19.119    9.559 flare-floss\floss\render\default.py:158(render_static_substrings)
506523/289779    0.333    0.000    8.042    0.000 {method 'extend' of 'list' objects}
289276/289012    0.375    0.000    6.782    0.000 flare-floss\venv\lib\site-packages\rich\console.py:1286(render)
   288911    0.435    0.000    6.004    0.000 flare-floss\venv\lib\site-packages\rich\text.py:646(__rich_console__)
    72213    0.257    0.000    5.433    0.000 flare-floss\venv\lib\site-packages\rich\console.py:1485(_collect_renderables)
    72213    0.244    0.000    5.302    0.000 flare-floss\venv\lib\site-packages\rich\console.py:978(options)
   216639    0.620    0.000    4.448    0.000 flare-floss\venv\lib\site-packages\rich\console.py:991(size)
    72296    0.140    0.000    3.722    0.000 flare-floss\venv\lib\site-packages\rich\console.py:1396(render_str)
    72296    1.135    0.000    3.582    0.000 flare-floss\venv\lib\site-packages\rich\markup.py:103(render)