search

s3team / Polyglot

MIT License
101 stars 17 forks source link

Missing seeds #3

Closed 1789120321 closed 1 year ago

1789120321 commented 3 years ago

Hello. The seed data you used is not found in the code. Would you like to share the seed data you used?

Hanseltu commented 3 years ago

Hello. Thanks for sharing such a nice tool.

Just following @1789120321's issue, would you like to update the seed data and the command line used in the experiment? Specifically, I am interested in how you test C compilers. Did you use the seed programs from GCC or LLVM test suits and how did you add the optimization options to the command line while testing?

Any hints or suggestions are welcome! Thanks and waiting for your reply.

Best wishes, Haoxin

Changochen commented 3 years ago

Hello. We are not going to share the seed inputs for now. But the seeds we use for testing are all from the test suits from the targets we test. For javascript we use https://github.com/tunz/js-vuln-db instead.

how did you add the optimization options to the command line while testing?

This is just the same as how AFL does it because PolyGlot is based on AFL.

Hanseltu commented 3 years ago

Hi @Changochen. Thanks for your reply.

Following your suggestions, I have tried using test suites from https://github.com/gcc-mirror/gcc/tree/master/gcc/testsuite/gcc.c-torture to start testing GCC compiler. But after around 40min running, I got an error from typesystem.cpp, and the assertion tells "afl-fuzz: ../src/typesystem.cpp:646: static void TypeSystem::collect_structure_definition(IR*, IR*): Assertion 'var_name' failed".

I have no idea what happened here. Did you have the same experience before? Or what kinds of code PolyGlot may fail to handle?

Thank you very much!

Changochen commented 2 years ago

Sorry for the late reply. I haven't checked the repo for a long time. I think this is because there is an input containing a struct with a struct member without name.

For example, 

struct X {
int;
};

PolyGlot assumes the initial seeds are correct and tries to produce correct mutated inputs, and it will filter the testcases it cannot handle during initialization. I only met this problem during development but not fuzzing. Have you modified the C grammar? If not it might be a bug in PolyGlot. One possible solution to fix this is to remove some of the incorrect seeds at the beginning.