cmu-sei / pharos

Automated static analysis tools for binary programs
Other
1.55k stars 190 forks source link

Generating JSON fails with duplicate key. #274

Open Raphtaliyah opened 2 hours ago

Raphtaliyah commented 2 hours ago

I've run OOAnalyzer on a large program (9MB) using the multi step guide and it fails at the last step when it's generating the JSON results:

ERROR: Duplicate key: '0xa8b408'
ERROR: In:
ERROR:   [20] with_output_to(<stream>(0x5596878b5520),exportJSON)
ERROR:   [19] setup_call_catcher_cleanup(user:open('Result.json',write,<stream>(0x5596878b5520)),user:with_output_to(<stream>(0x5596878b5520),exportJSON),_142444,user:close(<stream>(0x5596878b5520))) at /usr/local/lib/swipl/boot/init.pl:663
ERROR:   [16] catch(user:exportJSONTo('Result.json'),error(duplicate_key('0xa8b408'),context(...,_142542)),user:(...,...)) at /usr/local/lib/swipl/boot/init.pl:562
ERROR:   [15] catch_with_backtrace('<garbage_collected>','<garbage_collected>','<garbage_collected>') at /usr/local/lib/swipl/boot/init.pl:629
ERROR: 
ERROR: Note: some frames are missing due to last-call optimization.
ERROR: Re-run your program in debug mode (:- debug.) to get more detail.

running in debug mode: (I assume I had to add :- debug. to ooprolog? I'm very unfamiliar with these tools.)

ERROR: Duplicate key: '0xa8b408'
ERROR: In:
ERROR:   [20] with_output_to(<stream>(0x55a3e77b4590),exportJSON)
ERROR:   [19] setup_call_catcher_cleanup(user:open('Result.json',write,<stream>(0x55a3e77b4590)),user:with_output_to(<stream>(0x55a3e77b4590),exportJSON),_464,user:close(<stream>(0x55a3e77b4590))) at /usr/local/lib/swipl/boot/init.pl:663
ERROR:   [18] setup_call_cleanup(user:open('Result.json',write,<stream>(0x55a3e77b4590)),user:with_output_to(<stream>(0x55a3e77b4590),exportJSON),user:close(<stream>(0x55a3e77b4590))) at /usr/local/lib/swipl/boot/init.pl:666
ERROR:   [17] exportJSONTo('Result.json') at /usr/local/share/pharos/prolog/oorules/oojson.pl:403
ERROR:   [16] catch(user:exportJSONTo('Result.json'),error(duplicate_key('0xa8b408'),context(...,_662)),user:(...,...)) at /usr/local/lib/swipl/boot/init.pl:562
ERROR:   [15] catch_with_backtrace(user:exportJSONTo('Result.json'),error(duplicate_key('0xa8b408'),context(...,_734)),user:(...,...)) at /usr/local/lib/swipl/boot/init.pl:629

If I grep the results file for the duplicate key (0xa8b408) I get:

finalVFTable(0xa8b408, 0x4, 0x4, 0xa8b404, '.?AU?$error_info_injector@Vbad_lexical_cast@boost@@@exception_detail@boost@@').
finalVFTableEntry(0xa8b408, 0, 0x42d0f0).
finalClass(0xa8b408, 0xa8b408, 0x28, 0x28, 0x42d0f0, [0x42cd80, 0x42ce60, 0x42d0f0]).
finalEmbeddedObject(0xa8b408, 0, 0x9e28cc, likely).
finalInheritance(0xa8b408, 0xa882b0, 0x14, 0xa8b408, false).
finalInheritance(0xa8b408, 0xa8ae80, 0, 0xa8b408, false).
finalInheritance(0xa8b410, 0xa8b408, 0, 0xa8b410, false).
finalMemberAccess(0xa8b408, 0, 0x4, [0x42cda9, 0x42cdf1, 0x42ce69, 0x42ce70, 0x42ce7e]).
finalMemberAccess(0xa8b408, 0xc, 0x4, [0x42cdb2]).
finalMemberAccess(0xa8b408, 0x10, 0x4, [0x42cdb8]).
finalMemberAccess(0xa8b408, 0x14, 0x4, [0x42cdc3, 0x42cdf7]).
finalMemberAccess(0xa8b408, 0x18, 0x4, [0x42cdcf]).
finalMemberAccess(0xa8b408, 0x1c, 0x4, [0x42cdde]).
finalMemberAccess(0xa8b408, 0x20, 0x4, [0x42cde8]).
finalMemberAccess(0xa8b408, 0x24, 0x4, [0x42cdee]).
finalDemangledName(0xa8b408, '.?AU?$error_info_injector@Vbad_lexical_cast@boost@@@exception_detail@boost@@', 'boost::exception_detail::error_info_injector<class boost::bad_lexical_cast>', '').

I have the files from the previous steps and can send them in private if needed (the log file is 8.1GB tho and the first 3 steps took 48 hours to run and needed 155GB of RAM).

sei-eschwartz commented 2 hours ago

Can you share your results file? If so, that is the easiest way for us to see what is going on. You can email to eschwartz@cert.org if posting publicly is a concern.

It's a little odd that 0xa8b408 supposedly inherits from 0xa8ae80 at 0, and embeds 0x9e28cc at 0. But it's not impossible if 0xa8ae80 is a zero size class. And since 0xa8b408 is some boost exception class, that seems likely.

Raphtaliyah commented 1 hour ago

My concern was the name of the executable making this issue show up in search results but it should hopefully not this way! results.zip