intTests create many intermediate files not tracked by git

[RyanGlScott]

After running intTests/runtests.sh, my git tree is left in a significantly dirtier state:

$ git status
On branch java-bin-dirs-tweaks
Untracked files:
  (use "git add <file>..." to include in what will be committed)
        intTests/test0057/c.out
        intTests/test0057/t1.out
        intTests/test0057/t3.out
        intTests/test0057/t5.out
        intTests/test0057/x.out
        intTests/test_external_abc/offline_verilog.prove0.v
        intTests/test_external_abc/w4_offline_smtlib2.prove0.smt2
        intTests/test_external_abc/write_smtlib2_w4_sat.smt2
        intTests/test_external_abc/write_smtlib2_w4_unsat.smt2
        intTests/test_external_abc/write_verilog_sat.v
        intTests/test_external_abc/write_verilog_unsat.v
        intTests/test_intro_examples/mulcomm1.cnf
        intTests/test_intro_examples/mulcomm2.prove0.cnf
        intTests/test_llvm_x86_01/test.bc
        intTests/test_llvm_x86_02/test.bc
        intTests/test_llvm_x86_03/test.bc
        intTests/test_llvm_x86_04/test.bc
        intTests/test_llvm_x86_05/test.bc
        intTests/test_llvm_x86_06/test.bc
        intTests/test_profiling/prof/
        intTests/test_smtlib/prove.prove0.smt2
        intTests/test_smtlib/sat.smt2

nothing added to commit but untracked files present (use "git add" to track)

This makes making changes to the intTests directory more annoying, since you have to avoid accidentally git adding these files. It seems like most of these files are autogenerated during the course of running the integration tests, so we should audit each file and either:

1. Add it to a .gitignore file, or
2. Check it in to version control. Some of these files, such as .bc files, might actually differ between different LLVM versions / operating systems / etc., so checking them in might make these tests more portable. But this would need to be evaluated on a case-by-case basis.

[brianhuffman]

I think the appropriate thing to do in many cases is actually

3. Add a command to test.sh to delete the intermediate files after running the test.

Several integration tests already do this for temporary files that are created earlier in the same script.

In the case of .bc files, I think the answer is that those should always be checked in. Otherwise it can be hard to recreate the right file locally depending on which OS you're running. For the test_llvm_x86_* tests, we should probably also check in the compiled .o files; I think the ones I have in my local repo were copied from someone else who compiled them on Linux so that I could get the tests to work at all.

I'd say that we probably shouldn't be using .gitignore for anything left over from test scripts, unless we also provide a make clean script that can clean them up. For maximum reproducibility in the test suite, it would be best if it's possible to always run the integration tests with exactly the same initial directory contents.

[RyanGlScott]

I'd say that we probably shouldn't be using .gitignore for anything left over from test scripts, unless we also provide a make clean script that can clean them up.

That sounds reasonable to me.

[RyanGlScott]

My first inclination was to write a script that traverses each test*/ directory, and if there exists a Makefile with a clean target, run it. However, this is actually too aggressive for what we want, as there are some existing Makefiles where clean removes files that have been checked into version control. For example, intTests/test_crucible_jvm/Makefile's clean target will remove Arr.class, Dyn.class, etc.

Some other possibilities for a cleaning script:

1. Simply run git clean in each test* directory to remove its untracked files. This is also probably too powerful for what we want, as we'd like to specify exactly what files ought to be removed in order to avoid accidentally deleting our work when running the clean script.
2. Have separate clean targets in each target's Makefile, one of which is geared specifically towards the clean script (and therefore will never delete any version-controlled files), and another which is more aggressive (e.g., intTests/test_crucible_jvm/Makefile's clean target).

[RyanGlScott]

796 did some cleanup of the .gitignore situation in saw-script, so I'm inclined to just close this.