csmith-project / creduce

C-Reduce, a C and C++ program reducer
Other
1.47k stars 128 forks source link

Question: can C-Reduce support additional runnable scripts? #256

Closed Hanseltu closed 1 year ago

Hanseltu commented 1 year ago

Hi,

Thanks for the nice tool. I have used this for a long time to reduce compiler bugs. One issue that I encountered recently is when I reduced the C program for another purpose, the reduce.sh can return 0 when directly executing ./reduce.sh; echo $? but failed when I executed creduce --n 1 reduce.sh test.c.

The problem may come from the content in the reduce.sh, and the key portions are:

 gcc-12 -w -std=c99 -O1 test.c -o test1 &&\
 python run-test.py test1 > out1.txt &&\
 gcc-12 -w -std=c99 -O1 test.c -o test2  &&\
 timeout 2 ./test2 >& out2.txt &&\
 ! diff out1.txt out2.txt
$./reduce.sh ; echo $?
0
$creduce --n 1 reduce.sh test.c
C-Reduce cannot run because the interestingness test does not return
zero. Please ensure that it does so not only in the directory where
you are invoking C-Reduce, but also in an arbitrary temporary
directory containing only the files that are being reduced. In other
words, running these commands:

  DIR=`mktemp -d`
  cp /home/haoxin/haoxin-data/dut-research/se2ct-testing/1-test/test.c $DIR
  cd $DIR
  /home/haoxin/haoxin-data/dut-research/se2ct-testing/1-test/reduce_wrong_code.sh
  echo $?

should result in "0" being echoed to the terminal.

See "creduce --help" for more information.

I suspected that I can not get the correct return value mainly because the script run-test.py is not included in the temp file after DIR='mktemp -d', so creduce can not run it as usual. Is this the reason behind it? How can I add the new scripts (e.g., run-test.py) to the temp folder?

Thanks, Haoxin

eeide commented 1 year ago

Can you post your interestingness script in a comment here? If you make the script available to us, we can probably give you a more useful answer.

regehr commented 1 year ago

c-reduce's parallel mode is forced to run each test in a fresh temporary directory, or else different instances would tend to interfere with each other. you simply need to ensure that your interestingness test works when it is in a fresh directory. if you need something to be there (besides the file being reduced) then make sure that you copy it over (or, if it is immutable, refer to its original location using an absolute path)

Hanseltu commented 1 year ago

Understood. Thanks for your suggestions!