haf / expecto

A smooth testing lib for F#. APIs made for humans! Strong testing methodologies for everyone!
Apache License 2.0
663 stars 96 forks source link

Use dot as a separator instead of slash #364

Closed cmeeren closed 4 years ago

cmeeren commented 4 years ago

Originally posted in https://github.com/YoloDev/YoloDev.Expecto.TestSdk/issues/26, but I was redirected here.

When the VS test explorer renders the test tree, it seems to use dot as a separator (if grouping by namespace/class).

Consider the following tests:

[<Tests>]
let tests =
  testList "Domain" [
    testList "Username" [
      testList "create" [
        testCase "returns Error if input is empty" ignore
        testCase "returns Error if input length is > 100" ignore
        testCase "trims the input if Ok" ignore
      ]
    ]
    testList "CustomerNumber" [
      testList "create" [
        testCase "returns Error if input is empty" ignore
        testCase "returns Error if input is non-numeric" ignore
        testCase "returns Ok if the input is numeric" ignore
      ]
    ]
  ]

Test Explorer shows it like this:

image

All tests appear in a flat list. This makes it hard to read, hard to find relevant tests, and hard to select a related group of tests to run.

It would be much more useful if Test Explorer could show it as a tree. AFAIK this would only require using . instead of / as a separator in the hierarchy.

It can be worked around to some extent by adding dots to the end of each test list name:

[<Tests>]
let tests =
  testList "Domain." [
    testList "Username." [
      testList "create." [
        testCase "returns Error if input is empty" ignore
        testCase "returns Error if input length is > 100" ignore
        testCase "trims the input if Ok" ignore
      ]
    ]
    testList "CustomerNumber." [
      testList "create." [
        testCase "returns Error if input is empty" ignore
        testCase "returns Error if input is non-numeric" ignore
        testCase "returns Ok if the input is numeric" ignore
      ]
    ]
  ]

image

But it's not optimal: It requires changing the names, it still renders the slashes, and seems to only work for the last levels (e.g. Domain./CustomerNumber hasn't been split, and if there are single sibling test cases of nested tests lists, then they are not shown grouped in the same way).

If Expecto could support using . instead of / as a separator, I think that would give a much better experience in the VS test explorer. 🙂

Or if you have any other idea about how to improve this, that would be great!


Just to be clear about the sibling test stuff I mentioned, in case it's relevant:

[<Tests>]
let tests =
  testList "Domain." [
    testList "Username." [
      testCase "Some other username test" ignore
      testList "create." [
        testCase "returns Error if input is empty" ignore
        testCase "returns Error if input length is > 100" ignore
        testCase "trims the input if Ok" ignore
      ]
    ]
    testList "CustomerNumber." [
      testList "create." [
        testCase "returns Error if input is empty" ignore
        testCase "returns Error if input is non-numeric" ignore
        testCase "returns Ok if the input is numeric" ignore
      ]
    ]
  ]

image

haf commented 4 years ago

Solved in #366

cmeeren commented 4 years ago

Hooray! Looking forward to release 😁 But will #366 alone fix the issue? Based on very cursory reading of that PR, it seems like there is a new option with a default value of /. But the VS test runner (or YoloDev.Expecto) doesn't use that, right?

haf commented 4 years ago

@cmeeren This is not true, dot is now the default.

cmeeren commented 4 years ago

Nice! So that means it should automatically work once I update expecto, with no changes required on YoloDev's part?

haf commented 4 years ago
Screenshot 2020-04-04 at 11 43 58

Well, it's a major release, so there are breaking changes to the API, but we've kept them to a minimum. Let's see ;)?