Taint tracking tools instead of DFSan.

[spinpx]

Angora uses LLVM DFSan for taint analysis. Even it is the best taint tracking tool I can found, it is not friendly if your tested program has external libraries. Also, it needs source code to compile. I have tries PIN & Libdft. But it is too slow , and Pin 3.x can't use external libraries and libdft only supports 32bits (Vuzzer has migrated it to 64bits). Any suggestion?

[vanhauser-thc]

most PIN taint tracers work only with pin 2.14 which is so outdated its shocking. doesnt work well with 64 bit or 4.x kernels. (e.g. libdft, libdft64, triton, ...)

I developed a tain tracer on pin (3.x) and dynamorio, and dynamrio is about 20x (!) faster. plus it works on ARM and AARCH64. As my dynamorio taint tracer is WIP and so far only for ARM, it wont help you for Angora so far though.

my tain tracer based on PIN 3.x is x64 only, but also WIP and has a few bugs, so it does not work well :)

the downside of dynamorio is that it can be unstable as its mostly a constant in-development tool.

So: dynamorio: fast, many platforms. pin: reliable

vuzzer64 has ported libdft for 64 bit, but it is based on PIN 2.14.

So my recommendation - which would also bring the community the best way forward - porting libdft64 to PIN 3.x is the best solution as many libdft projects already exist and could (hopefully) seamlessly switch to an upgraded one.

However dynamorio would be a really good solution as well. but its more work.

[spinpx]

Thanks for your suggestion.

I have made libdft(32 bit version) support pin 3.x one years ago. I open source it on https://github.com/AngoraFuzzer/libdft-pin3 now. The code may be not work now because I have not tested and clean it. I decided to remove it because pin3.x use its own CRT. I have to re-implement the same code on it. Because of this, I was planed to use pin 2.x's libdft so that I can use my rust runtime. Your suggestion makes me considering using pin3.x now. Thanks again.

BTW, is https://github.com/vanhauser-thc/drtaint the tracer based on dynamorio ?

[spinpx]

Also, I found libdft do not support some instructions like SSE. It has bad performance in optimized programs. The NDSS 2019 paper One Engine To Serve’em All: Inferring Taint Rules Without Architectural Semantics mentioned it.

[vanhauser-thc]

@spinpx if you would have made your libdft for pin 3.x available when you wrote it, the vuzzer people would have 64-bit enabled that one and we would all be more happy :)

the libdft64 does not have sse support btw.

Referencing https://github.com/vusec/vuzzer64/issues/5

[vanhauser-thc]

@spinpx RE: BTW, is https://github.com/vanhauser-thc/drtaint the tracer based on dynamorio ?

no its not. this is by a italian student where I wanted to fix a bug. its also ARM, and its faster than mine (because its using special dynamorio shadow memory) but it lacked the flexibily I neede for my taint tracing. But for the purposes of angora drtaint could perhaps work well!

[spinpx]

https://github.com/vusec/vuzzer64/pull/4

VUzzer64 has an pull request (and merged) for pin 3.x.

[vanhauser-thc]

I tried vuzzer64's libdft64 with pin-3.7 - and its not working.

1) it has the command line options removed to assess opened files, etc. - so I rewrote a function so that it adds filedescriptors from a specific directory. 2) then I used unrar and m5sum on input files. 3) it correctly adds the fd of the input file and taints the input data 4) it never follows the accessed bytes and does not taint the wites to the output (stdout for md5sum and the output file for unrar)

Here is the relevant output of "unrar x /tmp/test.rar":

In open
 Inserted /tmp/test.rar 3.
 Setting taint 3 7 bytes
 Setting taint 3 7 bytes
 Setting taint 3 6 bytes
 Setting taint 3 7 bytes
 Setting taint 3 35 bytes
 In open
 in_dtracker_whitelist /usr/share/zoneinfo/Europe/Berlin
 Info ignoring fd 4 /usr/share/zoneinfo/Europe/Berlin
 Setting taint 4 2335 bytes  # why is it setting taint when the fd is ignored? bug ...
 Setting taint 4 1476 bytes  # why is it setting taint when the fd is ignored? bug ...
 close 4
 Setting taint 3 7 bytes
 Setting taint 3 35 bytes
 In open
 in_dtracker_whitelist /prg/tmp/vuzzer64/libdft64/test.txt
 Info ignoring fd 4 /prg/tmp/vuzzer64/libdft64/test.txt
 In mmap -1 0 bytes
 Setting taint 3 14 bytes
 close 4
 Setting taint 3 7 bytes
 close 3

[spinpx]

We have implemented the new feature for supporting libdft64 here. https://github.com/AngoraFuzzer/Angora/blob/master/docs/pin_mode.md