search

ldtri0209 / robotframework

Automatically exported from code.google.com/p/robotframework
Apache License 2.0
0 stars 0 forks source link

Smart monkey testing - provide way to run tests indefinitely, in "random" order based on state machine #1121

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
Smart monkey testing is a method of doing stress testing in which the monkeys 
know what state a program is in, as well as how to get from one state to 
another, but otherwise, decisions are made at random and the test can run 
indefinitely.

To implement this, each test or keyword should have a specified initial state 
and a specified final state. By doing this, the tests or keywords become the 
transitions in a state machine, and states (obviously) are the nodes.

Once we have this state machine and our initial state is defined, then the test 
should begin and traverse the state transitions (i.e., tests or keywords) at 
random. It would also help if we could have a good way of randomizing data as 
well as keywords. For example, instead of just allowing a default value for 
parameters, if we could specify a default range, or list, or values, then the 
test could pick a value at random and provide even better test coverage.

It would probably make sense to restrict this mode of "smart monkey testing" to 
tests or keywords marked with a specified tag, as it would probably only make 
sense for certain tests or keywords.

And perhaps the feature could be called "robotmonkey"!

Original issue reported on code.google.com by SamerA...@gmail.com on 1 May 2012 at 2:08

GoogleCodeExporter commented 9 years ago 
This seems somewhat related to 
http://code.google.com/p/robotframework/issues/detail?id=825

Original comment by SamerA...@gmail.com on 1 May 2012 at 2:15

GoogleCodeExporter commented 9 years ago 
Related tool: https://github.com/mkorpela/RoboMachine

Can not currently do online mbt - which is more or less equal to smart monjey 
testing. Offline mbt is still fairly close..

Original comment by mikko.ko...@gmail.com on 1 May 2012 at 4:29

GoogleCodeExporter commented 9 years ago 
Good idea but outside the scope of the core framework. Mikko already mentioned 
RoboMachine project that implements something similar. Adding this kind of 
functionality to it could be possible, but it would also be possible to start a 
new project for that purpose. You are right that issue 825 implementing this is 
easier once issue 825 is done.

Original comment by pekka.klarck on 2 May 2012 at 11:39

GoogleCodeExporter commented 9 years ago 
After making this request, I have since found out about two open-source tools 
that can do model-based testing with RobotFramework:

TEMA (http://tema.cs.tut.fi/index.html) seems very promising, and includes a 
library for RF. See related research paper describing its use with 
RobotFramework at 
http://www.cs.umd.edu/~atif/testbeds/TESTBEDS2011-papers/Pajunen.pdf 

MISTA (http://www.homepages.dsu.edu/dxu/research/MBT.html) also supports 
RobotFramework.

And of course RoboMachine, as mentioned by Mikko.

-Samer

Original comment by SamerA...@gmail.com on 2 May 2012 at 4:24