Our test suite can be improved. The current design has some limitations:

Problem 1: API

To run a test, you have to use the VMTest class interface provided and run your test like the following:

        def enter_NtOpenKey(syscall):
            pass

        def enter_NtCreateKey(syscall):
            pass

        hooks = {
            'NtOpenKey': enter_NtOpenKey,
            'NtCreateKey': enter_NtCreateKey,
        }
        events, exec_time = self.vm_test.run(cdrom_iso, hooks=hooks)

This forces you to use callbacks with all the problems that goes with it. For example it's impossible to maintain a state accross multiple callbacks. -> What if i want to get some information from one callback and use in another one ?

The solution to this would be to let the developer use the Nitro API,and provide VMTest class just as a convenient wrapper.

Nitro API:

with Backend(domain, analyze_enabled) as backend:
        backend.nitro.set_traps(True)
        for event in backend.nitro.listen():
              syscall = backend.process_event(event)
              if syscall.name == 'NtOpenFile':
                  # test

Problem 2 : Configurability

test_nitro.py has been written just to test Windows_7_x64 and nothing more.

As a result, the vm name is hardcoded in the setUp:

class TestNitro(unittest.TestCase):

    def setUp(self):
        con = libvirt.open('qemu:///system')
        domain = con.lookupByName('nitro_win7x64')
        self.vm_test = VMTest(domain)
        self.cdrom = CDROM()
        # clean old test directory
        test_dir_path = os.path.join(os.path.dirname(os.path.realpath(__file__)), self._testMethodName)
        shutil.rmtree(test_dir_path, ignore_errors=True)
        os.makedirs(test_dir_path, exist_ok=True)
        self.script_dir = os.path.dirname(os.path.realpath(__file__))
        # chdir into this directory for the test
        self.origin_wd = os.getcwd()
        os.chdir(test_dir_path)
        # create logging file handler
        self.f_handler = logging.FileHandler('test.log', mode='w')
        logging.getLogger().addHandler(self.f_handler)
        logging.info('Starting test at {}'.format(datetime.datetime.now()))

As we want to make tests on Windows 7 32 bits, as well as Windows 8, Windows 10 and for Linux, we have to find a may to make it configurable

Problem 3 : Split the tests

The test have only been designed for windows_7_x64 and as such, some of them will not run on a Windows_7_x86. On the other hand, some tests are generic enough to be run accross all Windows OS, and even on Linux

We have to group the tests that are generic enough and split the other in separate test modules, that will be called according to the configuration specified (either command line or a config file)

KVM-VMI / nitro

Improving the test suite #9

Problem 1: API

Problem 2 : Configurability

Problem 3 : Split the tests