Interpret the DeepBinDiff results

[ai-sta-website]

Hello,

I am writing to inquire some advices to interpret the output of DeepBinDiff. In particular, I have two questions as follows:

1. the processing time seems too large. I use the following command:

➜  DeepBinDiff git:(master) ✗ python3 src/deepbindiff.py --input1 experiment_data/coreutils/binaries/coreutils-7.6-O0/true --input2 experiment_data/coreutils/binaries/coreutils-7.6-O3/true --outputDir output/

And the processing time is:

python3 src/deepbindiff.py --input1  --input2  --outputDir output/  63233.15s user 103785.18s system 1966% cpu 2:21:33.48 total

It takes quite a long time (we are running it on a 32-core server machine with 256GB RAM). Is it normal?

2. The output to compare true vs. true is as follows:

Reading...
time:  7696.551887512207
Saving embeddings...
Perform matching...
[[0.8654591  0.92791235 0.7441185  ... 0.9215279  0.9736992  0.97301173]
 [0.74939525 0.8753574  0.6855561  ... 0.9378971  0.95241654 0.9951242 ]
 [0.6706515  0.82596886 0.8066987  ... 0.805171   0.9380803  0.9999579 ]
 ...
 [0.         0.         0.         ... 0.         0.         0.        ]
 [0.         0.         0.         ... 0.         0.         0.        ]
 [0.         0.         0.         ... 0.         0.         0.        ]]
(1044, 1044)
matched pairs:
[[161, 875], [164, 867], [84, 828], [389, 1309], [71, 811], [346, 1287], [302, 1212], [208, 987],
 [90, 833], [74, 814], [218, 1292], [467, 1556], [110, 844], [91, 1102], [456, 1562], [279, 1196]
, [75, 816], [213, 1317], [264, 1166], [77, 815], [76, 819], [102, 834], [291, 1206], [70, 856],
[329, 1248], [602, 1578], [560, 1581], [455, 1584], [692, 1543], [222, 999], [375, 1301], [392, 1
218], [49, 789], [1, 809], [374, 1219], [635, 1724], [267, 1177], [458, 1541], [201, 977], [250,
1139], [597, 1699], [410, 1338], [257, 1154], [341, 1319], [244, 1161], [248, 1100], [203, 978],
[734, 1920], [546, 1644], [598, 1696], [304, 1327], [372, 1302], ...

May I ask how to interpret the results? I am familiar with BinDiff and expecting similar output format like BinDiff (function-level and binary-level similarity). Is it possible to covert the current output into function or binary-level similarity score? Thank you very much!

[yueduan]

Hi, currently our system is not multi-threaded. So the runtime heavily depends on the power of the core rather than the number of cores. I guess it could be possible that the diffing is slow. There are some workarounds you can try.

[yueduan]

1. reduce the number of steps and the threshold for loss in libnrl/tadw.py.
2. reduce k in get_k_neighbors() in utility.py

[yueduan]

For the diffing results, sorry we dont have the fancy GUI output as BinDiff. The diffing results are directly printed as 'matched pairs:'. And according to our design, we don't match functions. Thank you!

[ai-sta-website]

@yueduan I see. Thank you for your clarification. I am basically looking for a replacement of BinDiff since it does not perform well. I noticed in the paper, it is mentioned to instrument and use Asm2Vec and InnerEye for similarity analysis. May I ask how you setup the Asm2Vec tool? I only found a web server which seems not handy to use, and cannot be extended. Thank you very much!

[yueduan]

We ended up implementing a Asm2Vec by ourselves.