Current behavior from lobster-python count unittests as code

[DiFerMa]

I used lobster-demo to test, I added to software.py:

def hello():
   # lobster-trace: SomeIdentifier
   print("This is function!")   

def intValue():
   # lobster-trace: vanillaEg.OutputValue
   return 10

hello()
intValue()

I then in root folder created a unittests.py:

import unittest
from software import intValue

class TestIntValue(unittest.TestCase):
    def test_return_value(self):
        # lobster-trace: vanillaEg.OutputValue
        result = intValue()
        self.assertIsInstance(result, int)

    def test_return_value2(self):
        result = intValue()
        self.assertIsInstance(result, int)

if __name__ == '__main__':
    unittest.main()

Current html report's output with these commands: lobster-python . --out="python.lobster" and lobster-python . --out="unittests.lobster" --activity. This makes that the two test functions in unittests.py count as "code" making a total of four.

If file is specified, then it works better. The html report's output with these commands: lobster-python software.py --out="python.lobster" and lobster-python . --out="unittests.lobster" --activity looks as it should, with correct number of functions:

For documentation purposes I will paste also the contents of the other configuration files:

lobster.config

requirements "Requirements"{
 source: "trlc.lobster";
 source: "codebeamer.lobster";
}

implementation "code" {
  source: "python.lobster";
  trace to: "Requirements";
}

activity "unittests" {
  source: "unittests.lobster";
  trace to: "Requirements";
}

main.rsl

package vanillaEg

type Requirement{
    desc String
}

main.trlc

package vanillaEg

Requirement trlc {
  desc = "Some requirements text for type trlc, eg. this function must do X Y"
}

Requirement OutputValue {
  desc = "Return value from function must return an integer value"
}

lobster-trlc.conf

vanillaEg.Requirement{   
   description = desc
}

[phiwuu]

We could reproduce the issue, but currently it is not a bug. It works as designed. The current solution is to call lobster-python only on directories which do not contain tests.

Use Cases

However, we can talk about improving the behavior, and make it more intuitive. In general, we want to support the following scenarios:

1. Only code
2. code with tests in separate folder, and there are also utility functions used only by the tests (not by the production code)
3. Only code, but it contains functions that start with test (example: test_user_input could be a function that validates some user input, and is not a unit test)

Only Code

This scenario is simple and currently supported. No switch case is needed, since unit tests don't exist.

Code with Tests:

The assumption is that, all tests functions start with test, and hence can be identified through their names. Utility functions used by the unit tests are ignored, unless they start with test as well.

• The --activity flag has been introduced for exactly this use case to extract the test cases.
• lobster-codebeamer must be executed twice, once for tests and once for production code, and it is necessary to clearly separate code and tests into different folders.

Code with test in function names

This is the same as "Only Code". It is supported as long as no unit tests exist.

Possible Update

A possible update shall

• be backwards compatible
• make it easy to the user to read unit tests

So a possible feature could be to add a new argument called --implementation-only which ignores every function that starts with test.