Describe/Context/It with <var> syntax in it doesn't report state

[TylerLeonhardt]

Example:

Context "asdf <env:PATH>" {
}

This doesn't show any state of running because when we "discover" this test, it is asdf <env:PATH> but when we run this test, it is asdf /bin:/usr/local/bin:..... There's no concrete way of matching the discovery result from Pester with the test result output.

This does work for ones with "TestCases" because those are structured with a test-suite of asdf <Foo> (where Foo is a property on a test case) and test-case children of asdf ActualValueOfFoo

[TylerLeonhardt]

I think I need @nohwnd's thoughts on this to figure out a way to achieve this.

[Glober777]

Actually, @nohwnd has just released Pester 5.2, which allows running only the Discovery phase:

Discovery only You can now run only discovery. There is new option Run.SkipRun. Use it with Run.PassThru, to get the result object with all the tests in your test files. All tests will be marked as NotRun, unless there is Discovery failure of course.

That should solve the expansion issue as well as Dynamic tests (#44).

[nohwnd]

Discovery only might help here, but only for static data. At the moment you can't take the discovered tests and feed that back to Invoke-Pester to run it. It will do second discovery and that can differ from the first one. If the above template expands to current date time, you will still have different text for discovery only and the full run.

There's no concrete way of matching the discovery result from Pester with the test result output.

I think I don't understand this, and so I don't understand the whole problem. But when I was trying to match discovery results to run results I used Path + scriptblock startline as an id, that is pretty unique unless you have multiple tests on one line (and in that case you need the column as well. But it seems like you are after matching a given Describe + data, to a given Describe + Data, which would only work if they were from the same discovery. In that case you could use the path+line to get the correct descibes, and then .data == .data to get the actual "instance" of that describe.

But again maybe I am just misunderstanding the problem.

[nohwnd]

After reading some other issue that links to this I think I know what the problem is, and there is no generic solution for this.

Problem:

You get list of objects that represent the discovered tests, some It blocks are generated from -TestCases / -ForEach. You run tests. You get list of objects that represent the executed tests. You don't know how to map the results of the executed tests to the discovered tests.

Or in other case:

You have discovered tests. You run subset of them. You get results. You don't know how to map those, because some tests are generated by TestCases and they might have changed in the meantime.

Solution: I don't have one. Tests can be generated in dynamic way, and even if they are not, there is no persistent ID that we can give you. So there is no way to map results reliably. Especially in a case when you would run a test case inside a block that has -ForEach, that would give you multiple results in two separate parts of the tree. Best I can do is add an index everywhere where data are generated. And you need to be good at detecting changes in files and re-discovering.