How best to test client-server and network behaviors

[GoogleCodeExporter]

I've written most dots unit tests to use particular machine settings that
make a singe instance of Matlab act as client and server.  This is a good
test configuration since it exercises important code paths without
involving any physical network.

But it will also be important to test network behavior explicity.  Indeed,
there should be some tests, like TestDotsAllRemotesConfigIndependent should
be able to pass under several configurations.

It's awkward to try to include network-dependent unit tests among the
regular test/Test* which get invoked by runAllDotsTests, since some are
likely to fail without external setup (i.e. starting the client).  So maybe
there should be a separate test suite to be run following explicit client
setup...

Original issue reported on code.google.com by Benjamin.Heasly on 21 Nov 2009 at 5:54

[GoogleCodeExporter]

I've used a machine settings file to load configurations required for some 
tests. 
But I don't like this approach because the file can so easily, and silently, go
stale, and introduce stupid test errors.

It should be better to just set system modes as required for each test.

Or, perhaps a settings file is allowed to be partial, and supplemented by 
defaults.

Original comment by Benjamin.Heasly on 11 Dec 2009 at 5:42

[GoogleCodeExporter]

The runAllDotsTests script now takes arguments which can set system modes or 
other convenience test-specific modes, like a look-see for graphics.

A TestCase subclass, DotsTestCase, reapplies these values for each test.  It 
remembers them with global variables.

Thus, the tests can be run under different configs, *and* the 
MachineConfiguration 
and Switchboard can be cleared and reset to undergo their own tests.

As of 53 (and somewhat earlier) this approach has worked and allowed the same 
test 
suites to pass under various network configurations, on my laptop as well as at 
the 
Gold lab.  So it's good.

Original comment by Benjamin.Heasly on 16 Feb 2010 at 10:36

• Changed state: Fixed