SmallSuiteGenerator

SmallSuiteGenerator is an Smalltalk tool to generate test cases automatically

MIT Licensed.

SmallSuiteGenerator can be used to discover bugs in the code, because it generates automatically test cases

1. Installation

1.1. Pharo

To install the latest stable version of SmallSuiteGenerator in a Pharo image, execute the following code:

Metacello new
 baseline:'SmallSuiteGenerator';
 repository: 'github://OBJECTSEMANTICS/SmallSuiteGenerator:master/src';
 load.

To install the version that presents graphic results of evolution coverage, execute the following code:

Metacello new
 baseline:'SmallSuiteGenerator';
 repository: 'github://OBJECTSEMANTICS/SmallSuiteGenerator:master/src';
 load: #('All').

To install the version that allows SmallSuiteGenerator to be exported to VisualWorks, run the following code:

Metacello new
 baseline:'SmallSuiteGenerator';
 repository: 'github://OBJECTSEMANTICS/SmallSuiteGenerator:master/src';
 load: #('Exporter').

1.2. VisualWorks

See documentation in Install FuzzyTester

2. Configuration

There are two ways to do the automatic generation of tests, in the first way we use the automatic generation of this class, and in the second we define the configuration in a class generated by ourselves.

Define the typeInfo. You must first define the block of code that will be instrumented to get the typeInfo. Also define the regular expression pattern for the package.

| typeInfo aBlock |
aBlock := [ 
            (SSTeacher name: 'Ann' with: 50)
            nickname;
            canRegister: ((SConference price: 50) offerPrice: 50);
            idTeacher;
            yearsWorkExperience ].
typeInfo := STypeInfo asTypeInfo: (
        SSTypeCollector profile: aBlock onPackagesMatching: 'SmallSuiteGenerator-Scenario').

2.1. Generate tests from default

The STestCaseFactor class allows you to create a default test class, for this you must first provide the name of the targetClass, here is an example of this.

(STestCaseFactoryPharo new)
    targetClassName: #SSTeacher;
    typeInfo: typeInfo;
    seedBlock: aBlock;
    createTestCases;
    yourself.

You can also modify some default values for the generation of tests, an example is presented below.

(STestCaseFactoryPharo new)
    targetClassName: #SSTeacher;
    typeInfo: typeInfo;
    targetPackageRegex: 'SmallSuiteGenerator-Scenario';
    outputPackageName: 'Generated';
    numberOfGenerations: 10;
    numberOfStatements: 30;
    populationSize: 20;
    seedBlock: aBlock;
    createTestCases;
    yourself.

After to execute this script, the testCases will be generated with the higher fitness of the population. In the output package name specified you can find the generated testCases. Usually the last enumerated testCases have the highests fitness.

In case you have loaded the version with visualization and you want to generate the evolution graphs, execute the generateViews method beforecreateTestCases.

Hint: The methods to visualize only are available if you install the version with graphics, specified before.

2.2. Generate tests from a configuration class

Define configuration class. To do this, you must create the configuration class for the class you want to generate tests, to do this, execute the following command with the class you want, in this case we will create the configuration class for "SSTeacher" class.

SSTeacher createTestCaseConfig

With the script above, a class with the name "GASSTeacherTest" is automatically created in "GeneratedTests" package, which contains a series of default settings that inherits from "SConfigGenerationTestCase" superclass.

In case you prefer to generate a configuration class with a predefined name, you can run the following script:

SSTeacher createTestCaseConfigWithName: 'SSTeacherTest'

Change the default settings to your preference. Here are the most common settings that can be modified:

targetPackagesRegex: Regular expression of the package where the class is found.
```
GASSTeacherTest targetPackageRegex: 'SmallSuiteGenerator-Scenario'
```
outputPackageName: Package name where this class moved (configuration class).
```
GASSTeacherTest outputPackageName: 'Generated'
```
numberOfGenerations: Number of iterations that the genetic algorithm will perform.
```
GASSTeacherTest numberOfGenerations: 10
```
numberOfStatements: Maximum number of statements that the generated tests will have (without counting those statements that are assertions).
```
GASSTeacherTest numberOfStatements: 20
```
populationSize: Number of tests that will be generated.
```
GASSTeacherTest populationSize: 25
```
setUpMethod and tearDownMethod: In case the class from which you want to generate tests needs some necessary configuration before and after executing each test, it is necessary that you define these methods.

(STestCaseFactoryPharo from: GASSTeacherTest) 
    setUpMethod: 'setUp
    ^ self';
    tearDownMethod: 'tearDown
    ^ self'

fitness: Metric with which the coverage of the generated tests will be measured, there are 3 options: #statement, #method and #multi. STATEMENT measures the number of statements that the tests cover, METHOD measures the number of methods that are executed in the tests and finally MULTI is a unified statement and method metric.

GASSTeacherTest fitness: #statement

stopIterations: It is the number of iterations that is limited to expect the best fitness to improve, if not, the iterations are stopped.
```
GASSTeacherTest stopIterations: 5
```

Save and assign the typeInfo. Using the previously defined typeInfo, you can save this typeInfo in the configuration class, this in order to easily reuse the typeInfo and avoid generating it each time a different experiment is performed. To save typeInfo, use the following code

GASSTeacherTest addTypeInfo: typeInfo withKey: 'steacher'

Now, you need to reference the typeInfo by means of its key in the configuration class. Finally, we will assign the typeInfo that we save.

GASSTeacherTest typeInfo: 'steacher'

Generate tests. Below is an example of how to generate tests from this configuration class.

(STestCaseFactoryPharo from: GASSTeacherTest)
    seedBlock: aBlock;
    createTestCases;
    yourself.

Advanced settings

All these settings should be done before generate test cases

Sometimes the number variables is limited, to fix this problem you can change the default generations testing with variables by using a dictionary.

GASSTeacherTest asDict: true.

Examples

|typeInfo aBlock generated |
aBlock := [SFoo new
                return: Dictionary new;
                return: OrderedCollection new;
                return: (SSTeacher name: 'Ana' with: 11);
                returnFloat;
                returnString;
                returnCollection;
                returnNum;
                score;
                score: 5].
typeInfo := STypeInfo asTypeInfo: (
        SSTypeCollector profile: aBlock onPackagesMatching: 'SmallSuiteGenerator-Scenario').

generated := SStack createTestCaseConfigWithName: 'SSFooTestGenerated'.

generated addTypeInfo: typeInfo withKey: 'sfoo';
    typeInfo: 'sfoo';
    fitness: #statement;
    numberOfGenerations: 10;
    numberOfStatements: 30;
    populationSize: 20;
    stopIterations: 5;
    targetPackageRegex: 'SmallSuiteGenerator-Scenario';
    outputPackageName: 'Generated'.

(STestCaseFactoryPharo from: SSFooTestGenerated )
    generateViews;
    seedBlock: aBlock;
    setUpMethod: 'setUp
    ^ self';
    tearDownMethod: 'tearDown
    ^ self';
    createTestCases;
    yourself.

|typeInfo aBlock generated |
aBlock := [SFoo new
                return: Dictionary new;
                return: OrderedCollection new;
                return: (SSTeacher name: 'Ana' with: 11);
                returnFloat;
                returnString;
                returnCollection;
                returnNum;
                score;
                score: 5].
typeInfo := STypeInfo asTypeInfo: (
        SSTypeCollector profile: aBlock onPackagesMatching: 'SmallSuiteGenerator-Scenario').

(STestCaseFactoryPharo new)
    targetClassName: #SFoo;
    typeInfo: typeInfo;
    targetPackageRegex: 'SmallSuiteGenerator-Scenario';
    outputPackageName: 'Generated';
    numberOfGenerations: 10;
    numberOfStatements: 30;
    populationSize: 20;
    seedBlock: aBlock;
    generateViews;
    createTestCases;
    yourself.

Install SpyLite

See documentation in Install SpyLite

Install TestRunner

See documentation in Install TestRunner

OBJECTSEMANTICS / SmallSuiteGenerator

readme