some pdfs delay static analysis indefinitely

[mallorybobalice]

Hi, from a project using peepdf. https://github.com/spender-sandbox/cuckoo-modified/issues/54 Some samples are here

[mallorybobalice]

Well indefinitely or for say several minutes. 2 3 10 etc.

https://github.com/spender-sandbox/cuckoo-modified/blob/master/modules/processing/static.py

To see how PDFParser is used

[jesparza]

Hi there,

I am struggling to find some time to go through the issues. I hope I can find some time soon. Meanwhile, some comments and suggestions:

• If the PDF is not huge, probably the problem is the JS analysis. With option -m you void emulating JS code automatically. This can help when there are heap spraying loops, for instance.
• If the problem is the PDF parsing process (normally parsing huge documents), then the solution is more complicated as it would be more a problem in the way the PDF documents are parsed. I did not think too much in performance when I first coded the tool. Maybe if someone could do a profiling to see where it is hanging would help to locate the problem. Then we can try to check from that point.

Thanks for raising the issue!

Jose

[mallorybobalice]

I might add a traceback print on interrupting static analysis in cuckoo, to help narrow down the function Has anyone had time to look at it?

Size varies seen both small say 100kb, med 1mb and large 5mb pdfs do this , pretty sure saw references to js in old trace backs ?

emulating JS code automatically.

Mmmm, re -m option, could you please have a look above how cuckop static invokes peepdf and suggest how to make it do the equivalent of -m?

[jesparza]

I have taken a quick look with a profiler while analyzing the document with hash 66a35d6660059dd99d4abef26e9d1a3d (A_DOUT_SEC_12_10_2015.pdf). Some comments about that:

• Execution with -fl: The flag -l forces too many calls to re.findall, spending 50% of the time there. These calls are located in PDFParser.getIndirectObjects. It is also interesting to see that the next most expensive functions are related to js-beautify. Here I guess peepdf thinks there is JS in the document (bad JS detection in JSAnalysis.isJavascript) and that's why it calls analyseJS and therefore beautify.
• Execution without flag -l: The time gets reduced to the half of it.

Keep in mind that peepdf was designed to analyze malicious files, so normally small documents. This document is a 2MB-document. Not sure if with the current design (using re.findall) the time can be better than this. Maybe some things to check:

• See if the function PDFParser.getIndirectObjects can be improved performance-wise, especially when using -l. According to the profiling the function findall is called more than 90k times there...
• Try to improve the function JSAnalysis.isJavascript to detect better JS code. Keep in mind that this must be tested with malicious files too, not just benign ones. I knew already that this function was giving false positives, but I prefer FP than FN ;)

I hope this helps!

[jesparza]

Brad fixed the problem described in the first point at the end of the previous comment:

https://github.com/spender-sandbox/cuckoo-modified/commit/1d3bdee75006f792cf903baa24547cf2692f807c

After testing this and being sure that the modification is not missing anything we can include the code to fix the issue.

[spender-sandbox]

Will need to apply this as well: https://github.com/spender-sandbox/cuckoo-modified/commit/7e2f5211accf2c0600b8e193d4bd1eb915e53270

[Nwinternights]

@jesparza can you please apply that fixes? regards