search

haf / expecto

A smooth testing lib for F#. APIs made for humans! Strong testing methodologies for everyone!
Apache License 2.0
668 stars 96 forks source link

Add a message string to pending tests #265

Open 0x53A opened 6 years ago

0x53A commented 6 years ago

It would be nice if I could specify why a test is skipped (pending).

Maybe I'm waiting on a specific bugfix (issue tracker url), maybe the test is WIP, maybe the functionality under test is WIP.

MecuSorin commented 5 years ago

My approach to this development practice is to have a convention about the test naming. Is the simplest thing that solves your problem.

If you want to get fancy you can generate code that is consuming an issue tracker API to get a list of tickets, and create a wrapper over tests that takes as parameter such a ticket and adjust the test name accordingly. (you can gain insight using the Expecto tests filter)

In my opinion we should not change the attributes to support this (add complexity that is too particular).

@0x53A Can you, please, elaborate on your scenario, maybe I am missing something

0x53A commented 5 years ago

Sure. There are mainly three reasons to skip a test:

1) The test itself is WIP 2) The feature under test is WIP 3) The test is failing, you know it is failing, but want/need a green build anyway.

If you have a test results file, then it will contain something like

564 tests OK
2 tests skipped:
   1) test_something
   2) test_something_else

Now what I would like:

564 tests OK
2 tests skipped:
   1) test_something: Reason: "https://github.com/MyCompany/MyProject/issues/12345"
   2) test_something_else: Reason: "https://github.com/MyCompany/MyProject/issues/5678"

As you said, this can also be embedded in the test name.

Then I would have to rename my test to test_something_else__gh_issue_5678,

The goal is to add a skip reason that is easily accessible without navigating to the source code.