maester365 / maester

The core repository for the Maester module with helper cmdlets that will be called from the Pester tests.
https://maester.dev
MIT License
356 stars 87 forks source link

Duplicate tests that are 'NotRun' #472

Closed magnusjak closed 1 month ago

magnusjak commented 2 months ago

I couldn't find anything related to this that didn't seem to be specific to Github Actions.

I am running Maester through PowerShell locally, and I am getting this type of result on one of the tenants I am testing. I am just using the default tests that come out of the box (updated today), and not using any tags/path parameters. I am running version 0.3.0. If I open any of these duplicate tests, there's no info on why it isn't run, and all look exactly the same. Is there something I am doing wrong here, or is this some kind of bug?

image

I can see that the number of tests are caused from the number of $EmergencyAccessUsers. But would it be possible to skip the entire test entirely, instead of testing the actual skip/notRun within the foreach loop? :D

merill commented 1 month ago

Can you check your folder where you are running invoke-maester that you don't have duplicate tests.

Maybe trying giving direct path to the folder that has the test and changing into the folder that has the tests.

magnusjak commented 1 month ago

There's no duplicate tests, but as I mentioned the test loops through all $EmergencyAccessUsers, even if the test itself is skipped, it is then skipped $EmergencyAccessUsers.count number of times 😢

You can see this on line 26 in Test-ConditionalAccessWhatIf.Tests.ps1

Would it be possible to skip the entire test entirely, instead of testing the actual skip/notRun within the foreach loop? :D

merill commented 1 month ago

The what if tests are skipped by default since the What If API is still in preview. So you shouldn't be seeing these tests run as long as you are not including the CAWhatIf tag.

CleanShot 2024-10-21 at 08 13 52@2x