Run tests from the command line

[artemdinaburg]

We should be easily able to run tests from the command line. The documentation should show how to run the tests.

The tests exist, but they are kind of hidden. Looks like you have to do something like:

artem@nessie:/store/artem/ve/git/remill/tools/mcsema/tests/test_suite$ TRAILOFBITS_LIBRARIES=/store/artem/ve/git/remill/remill-build/libraries ./start.py

Some of the tests are failing for me (nc_amd64, the ada tests, xz), and I don't know if its because something is wrong, or because I didn't run them properly. Log below:

Summary

   The following tests have failed:
    > ada_trinary (linux/amd64)
      ---
      Command line:
      /store/artem/ve/git/remill/remill-build/libraries/llvm/bin/clang -rdynamic -o /tmp/ada_trinary /tmp/ada_trinary_amd64_linux.bc /usr/local/lib/libmcsema_rt64-4.0.a -Wno-unknown-warning-option -Wno-override-module -Wall  -lm -lgnat -lpthread -lgnarl -m64
      Exit code: 1
      stdout:
      The program exited with a value that is not zero: 1

      stderr:
      /tmp/ada_trinary_amd64_linux-3718d5.o: In function `sub_402892':
      /tmp/ada_trinary_amd64_linux.bc:(.text+0x22c3): undefined reference to `system__finalization_implementation_E'
      /tmp/ada_trinary_amd64_linux.bc:(.text+0x2351): undefined reference to `ada__finalization__list_controller_E'
      /tmp/ada_trinary_amd64_linux-3718d5.o: In function `__mcsema_constructor':
      /tmp/ada_trinary_amd64_linux.bc:(.text+0x6804): undefined reference to `ada__finalization__list_controller___elabs'
      /tmp/ada_trinary_amd64_linux.bc:(.text+0x6830): undefined reference to `system__stack_usage___elabs'
      /tmp/ada_trinary_amd64_linux.bc:(.text+0x6872): undefined reference to `system__finalization_implementation___elabb'
      /tmp/ada_trinary_amd64_linux.bc:(.text+0x68eb): undefined reference to `system__finalization_implementation___elabs'
      clang-4.0: error: linker command failed with exit code 1 (use -v to see invocation)

    > ada_worker (linux/amd64)
      ---
      Command line:
      /store/artem/ve/git/remill/remill-build/libraries/llvm/bin/clang -rdynamic -o /tmp/ada_worker /tmp/ada_worker_amd64_linux.bc /usr/local/lib/libmcsema_rt64-4.0.a -Wno-unknown-warning-option -Wno-override-module -Wall  -lm -lgnat -lpthread -lgnarl -m64
      Exit code: 1
      stdout:
      The program exited with a value that is not zero: 1

      stderr:
      /tmp/ada_worker_amd64_linux-1b391a.o: In function `sub_403072':
      /tmp/ada_worker_amd64_linux.bc:(.text+0x8541): undefined reference to `system__finalization_implementation_E'
      /tmp/ada_worker_amd64_linux.bc:(.text+0x85cf): undefined reference to `ada__finalization__list_controller_E'
      /tmp/ada_worker_amd64_linux-1b391a.o: In function `__mcsema_constructor':
      /tmp/ada_worker_amd64_linux.bc:(.text+0x97b3): undefined reference to `system__finalization_implementation__initialize'
      /tmp/ada_worker_amd64_linux.bc:(.text+0x97c9): undefined reference to `__gnat_rcheck_00'
      /tmp/ada_worker_amd64_linux.bc:(.text+0x9837): undefined reference to `system__tasking__protected_objects__entries__set_entry_name'
      /tmp/ada_worker_amd64_linux.bc:(.text+0x9879): undefined reference to `system__finalization_implementation__attach_to_final_list'
      /tmp/ada_worker_amd64_linux.bc:(.text+0x989a): undefined reference to `__gnat_rcheck_12'
      /tmp/ada_worker_amd64_linux.bc:(.text+0x98c6): undefined reference to `ada__finalization__list_controller___elabs'
      /tmp/ada_worker_amd64_linux.bc:(.text+0x98fd): undefined reference to `system__stack_usage___elabs'
      /tmp/ada_worker_amd64_linux.bc:(.text+0x993f): undefined reference to `system__finalization_implementation__finalize_list'
      /tmp/ada_worker_amd64_linux.bc:(.text+0x9976): undefined reference to `system__finalization_implementation___elabb'
      /tmp/ada_worker_amd64_linux.bc:(.text+0x9a1b): undefined reference to `system__finalization_implementation__limited_record_controllerIP'
      /tmp/ada_worker_amd64_linux.bc:(.text+0x9a26): undefined reference to `system__finalization_implementation___elabs'
      clang-4.0: error: linker command failed with exit code 1 (use -v to see invocation)

    > pthread1 (linux/amd64)
      ---
      Output:
      Create threads
      foo(), tls data=0 2
      bar(), tls data=0 2
      Wait for the threads to complete, and release their resources
      foo(), tls data=1 4
      bar(), tls data=1 4
      Main completed

      Expected:
      Create threads
      Wait for the threads to complete, and release their resources
      foo(), tls data=1 4
      bar(), tls data=1 4
      foo(), tls data=0 2
      bar(), tls data=0 2
      Main completed

    > gzip_amd64 (linux/amd64)
      ---
      Input: --version
      Exit code: -11
      stdout:
      The program exited with a value that is not zero: -11

      stderr:

    > nc_amd64 (linux/amd64)
      ---
      Input: -h
      Exit code: -11
      stdout:
      The program exited with a value that is not zero: -11

      stderr:

    > ada_thread1 (linux/amd64)
      ---
      Command line:
      /store/artem/ve/git/remill/remill-build/libraries/llvm/bin/clang -rdynamic -o /tmp/ada_thread1 /tmp/ada_thread1_amd64_linux.bc /usr/local/lib/libmcsema_rt64-4.0.a -Wno-unknown-warning-option -Wno-override-module -Wall  -lm -lgnat -lpthread -lgnarl -m64
      Exit code: 1
      stdout:
      The program exited with a value that is not zero: 1

      stderr:
      /tmp/ada_thread1_amd64_linux-6516e9.o: In function `sub_4025c2':
      /tmp/ada_thread1_amd64_linux.bc:(.text+0x2f1e): undefined reference to `system__finalization_implementation_E'
      /tmp/ada_thread1_amd64_linux.bc:(.text+0x2fac): undefined reference to `ada__finalization__list_controller_E'
      /tmp/ada_thread1_amd64_linux-6516e9.o: In function `__mcsema_constructor':
      /tmp/ada_thread1_amd64_linux.bc:(.text+0x638d): undefined reference to `ada__finalization__list_controller___elabs'
      /tmp/ada_thread1_amd64_linux.bc:(.text+0x63ae): undefined reference to `system__stack_usage___elabs'
      /tmp/ada_thread1_amd64_linux.bc:(.text+0x63e5): undefined reference to `system__finalization_implementation___elabb'
      /tmp/ada_thread1_amd64_linux.bc:(.text+0x6469): undefined reference to `system__finalization_implementation___elabs'
      clang-4.0: error: linker command failed with exit code 1 (use -v to see invocation)

    > xz_amd64 (linux/amd64)
      ---
      Input: --version
      Exit code: -11
      stdout:
      The program exited with a value that is not zero: -11

      stderr:

[artemdinaburg]

We can, now run the tests via the command line (we do it for Travis). You don't need IDA since we pre-make the CFGs.

keeping this open until there is something i can run that's akin to "scripts/run_tests.sh" that will find the environment variables for me and run the tests.