acceptance testing of a setup

[GoogleCodeExporter]

For complicated setups testing can be an issue.
You want to be sure everything still runs OK/unchanged.
(routes, channels, translations, mappings)

One of the problems is I/O: the I/O (paths, ftp-servers) in production are 
different from you test-environment.
(channels in production env are always different from test).

To accommodate this:
- extra option in bots.ini to turn this on '???'
- all channels use type=file
- channels have new attribute 'test-path'
- test-path will be used in communication (instead of path)

This allows you to:
- build a testset
- set value in bots.ini, slect routes, run
- results can be compared.
  This can be done automatically, by hand etc.
Probably it is a good thing to run a script. either per route, or one script.
script can compare etc.

- do all translations still run OK/unchanged?

Original issue reported on code.google.com by hjebb...@gmail.com on 10 Jan 2013 at 6:27

[GoogleCodeExporter]

Hi henk-jan,
I like this idea... normally I do it manually in my test environment, by 
changing channel type and path. But you do not always want this (eg. input from 
a database, testing a communicationscript...)

So the logic should be something like this? If channel has no testpath, make no 
change.
    if channeldict['testpath'] and botsglobal.ini.getboolean('settings','usetestpath',False):
        # over-ride the channel path and type for testing
        channeldict['path'] = channeldict['testpath']
        channeldict['type'] = 'file'
        botsglobal.logger.debug(u'Channel %s is using testpath %s',channeldict['idchannel'],channeldict['testpath'])

Kind Regards,
Mike

Original comment by mjg1964 on 11 Jan 2013 at 3:24

[GoogleCodeExporter]

about name of option: what about 'run_acceptance_test'?

test-script to compare results:
- nr files
- results
- file sizes
- content
- partners etc etc
test-script is run after whole acceptance-run because evaluate reeds to run 
first.

Original comment by hjebb...@gmail.com on 11 Jan 2013 at 2:01

[GoogleCodeExporter]

Original comment by hjebb...@gmail.com on 11 Jan 2013 at 4:59

• Changed title: acceptance testing of a setup

[GoogleCodeExporter]

commited changes in repository.
so for:
- counters/references are the same
- date/time strings are the same
uploaded a plugin which show how it works now.

Original comment by hjebb...@gmail.com on 14 Jan 2013 at 3:47

Attachments:

• plugin_acceptance_20130114.zip

[GoogleCodeExporter]

Original comment by hjebb...@gmail.com on 31 Mar 2013 at 11:47

• Changed state: Fixed

[GoogleCodeExporter]

Original comment by hjebb...@gmail.com on 10 Sep 2013 at 12:45

• Changed state: Done