search

croconut / godot-tester

A Github Action to handle testing Godot applications with GUT
MIT License
36 stars 14 forks source link

Coverage exceeds 100% when using parameterization tests. #4

Closed adarapata closed 3 years ago

adarapata commented 3 years ago

I'm using a translation tool, so sorry if it's hard to understand.

Thank you for making such a great product. 👍

When I include parameterization or perform multiple assertions in a single method, the rate becomes an inappropriate value.

Here is the current formula

rate = Tests / Passing asserets.

However, since passing asserets counts the number of assert methods, it may exceed the number of Tests.

I fixed it in the following way

https://github.com/adarapata/godot-tester/pull/1

However, this is not a complete solution because when a timeout occurs, it is not included in Passing asserts nor Failing asserts.

However, one problem has been solved and I am hoping that this will be taken care of once and for all. What do you think?

croconut commented 3 years ago

Thanks! And don't worry, it's completely readable 👍

Your solution looks like it solves that issue, would you like to pull request? I can either add in the timeout failing to your code or in another branch, up to you.

You'll also want to include local tests for it to run to ensure that it stays fixed.

croconut commented 3 years ago

I'm not exactly sure I'll be able to do much more with timeouts though. The process is being killed when it hangs / times out, so you won't be able to get test information as I'm unaware of a way to tell GUT to stop and also give me it's current information. It's more so a failsafe to prevent the action from eating up too much server time.

This seems like a good idea to bring up with the GUT author!

croconut commented 3 years ago

Okay, I have another way to do it.

Please, let me know if this fixes your issues, it is now released as v2.1a

adarapata commented 3 years ago

It worked fine. Great!

Thanks for the support.