search

se2p / pynguin

The PYthoN General UnIt Test geNerator is a test-generation tool for Python
https://www.pynguin.eu
MIT License
1.22k stars 74 forks source link

How to make pynguin continue even when coverage reach to 100% #38

Closed exKAZUu closed 1 year ago

exKAZUu commented 1 year ago

I want to generate various test inputs, but pynguin seems to stop when coverage is 100%. (I plan to use pynguin to generate robust test cases from the code of the model answer in order to evaluate code submitted by various users.)

StoppingConfiguration provides maximum options, but there are no minimum (at least) options. Is there a way to keep pynguin continue?

stephanlukasczyk commented 1 year ago

Sorry for the late reply.

Actually, there currently is no easy way of achieving this, except directly modifying the code of Pynguin's test-generation algorithms. I've just checked and each algorithm has a hard-coded early exit of the generation loop, as soon as 100% coverage is reached (e.g. https://github.com/se2p/pynguin/blob/083c49677c55b5f94af619c25f727b5ff6ed8d63/src/pynguin/generation/algorithms/randomteststrategy.py#L55, https://github.com/se2p/pynguin/blob/083c49677c55b5f94af619c25f727b5ff6ed8d63/src/pynguin/generation/algorithms/dynamosastrategy.py#L75).

If you change the source code of Pynguin and remove the respective check (the second predicate in the and conjunction, just leaving the condition to the while loop calling self.resources_left()), this should be doable. Maybe, at this point, I should add a note to my list of future features for Pynguin that this might be a good one.