Generate test cases for specified types only

[xylankant]

I'm looking for a tool to automatically generate tests for python functions, and came across pynguin. It looks really cool, and I was able to install it and run the tests from your docs.

However, when generating test, I notice that a lot of the time, pynguin seems to "ignore"(?) the type hints for function signatures. E.g., even for the triangle example where the hints specify x, y, z to be integers, many (the majority of) tests set some or all of these to be None types, or Booleans, or sometimes other datatypes.

I am wondering, is there an option to restrict test cases to using only types specified by the functions? I could not find anything like that in the documentation.

[stephanlukasczyk]

Thank you for you interest in Pynguin.

There is currently no option to restrict the type selection. This sounds interesting and like a reasonable feature. However, when we designed Pynguin, we had in mind that most Python code might not have type information annotated. In this case such a restriction would not be helpful at all. So, for now, there is no way to achieve the proposed—only use the annotated types.

There is, by the way, another reason to generate different types than the annotated ones: we've seen many code snippets that contain some guarding if conditions with type checks, such as if isinstance(parameter, str): that provide a specific handling for unexpected parameter types. Pynguin would miss such guarding conditions if it would only generate the annotated types.

Finally, I have to admit that there are many different scenarios one might want to use Pynguin. In the best case, it provides solutions for all of these. We'll make our minds on whether and how we could resolve your proposal in a future version of Pynguin.

[Wooza]

Maybe as an addition to what @stephanlukasczyk said: You can use --none-weight 0.0 --any-weight 0.0 to prevent Pynguin from using arbitrary values as arguments, though that still might not be enough.

[xylankant]

Sorry for commenting on a closed issue. Thank you for your answers, I think it could be a nice feature, but understand it's definitely not of a high priority. As for setting --none-weight 0.0 --any-weight 0.0 , this doesn't work unfortunately, e.g., the tests still generate some boolean inputs for integer signature.

I'll see if I can find a workaround, thanks for this cool project!

[Wooza]

Just as a side note: Using a bool argument for an int parameter is valid, for example, isinstance(True, int) holds.

[xylankant]

You are right of course, as bool inherits from int in python, and False/True are the same as 0/1. It would still be nice to have a "strict" typing observed in test cases, but yes, technically it's correct :)