Enable TreatNoTestsAsError in our test run configurations

[weshaggard]

As part of investigating https://github.com/Azure/azure-sdk-for-net/issues/26881 it was discovered that some tests were not running because of assembly load issues. Some of those issues were fixed with https://github.com/Azure/azure-sdk-for-net/pull/29127 however there are still more test projects that are failing to load and thus not running any tests but they are silently failing today. We should enable TreatNoTestsAsError option in our nunuit.runsettings as commented out in https://github.com/Azure/azure-sdk-for-net/pull/29127 to make sure these no longer silently fail. However before we can do that we need to fix the existing test projects so we don't have a ton of test failures right from the start.

To get a list of the currently broken tests one can run our kusto query:

https://dataexplorer.azure.com/clusters/azsdkengsys.westus2/databases/Pipelines?query=H4sIAAAAAAAAA22MOw7CMAyGd07xKxOcgI2BuXSCAxhiEktJjWJXVRGHJwudmL/HeZYSB02DTLz7YMncGFepbE71hRMo6f4YDxu7sBklRufNbRHPCKPCe4BK/shsAdr+egDG2ySOJ0nhCFcUpRj63OZaqcmbcV9/7RdvhKE2nAAAAA==

BuildLogLine
| where Timestamp > ago(7d)
| where Message startswith "No test matches" or Message startswith "   NUnit failed to load"
| summarize by Message

Here is some of them:

[pallavit]

Looked at the current failures and see the following

BuildLogLine | where Timestamp > ago(7d) | where Message startswith "No test matches" or Message startswith " NUnit failed to load" | where Message contains "Azure." and Message !contains "Microsoft.Azure" | summarize by Message, BuildDefinitionName

There are failures in storage, monitor, identity and EventHubs. @christothes could you please take a look at the identity failure here? Please use the following link to get more information - https://dataexplorer.azure.com/clusters/azsdkengsys.westus2/databases/Pipelines?query=H4sIAAAAAAAAA22OsQ7CMBBDd77CygQSzExlqBhLJ%2FiAg1zbk5oEJVdVRXw8yUCnbpbsZ7ueZLRN6BvxvPtiHjgy7uI4Kbk3LqA%2B7M%2F2sHo3Tol6Rvajpll0gGkDNANwpK%2BBk0GImzkA7cOLoiMZ2UIDxkDWrOV1eXPlTnJIgm%2FJMaoKxrPiBLHsVXTJssylwqXJOYryYTyXLfz4P%2FIDr6SgkOkAAAA%3D

@jsquire , @m-redding the eventhub failure can be found at - https://dataexplorer.azure.com/clusters/azsdkengsys.westus2/databases/Pipelines?query=H4sIAAAAAAAAA22OMQ7CMBAEe16xcgUS1FShQLSkIg%2B4xJf4pNhG9oUIlMdjJKBKt9LsrHbB7DgxbuI5K%2Fk7TqAhbo92t1m%2B7Mo508AoPGmeRR1MHaFFgCftHGeDmFZ7AOomiKInGdlCI8ZI1vzHz5OM9sK9lJLEUJNnVBVMYMUB%2FOCgbmpL7OQj5cl7SvJitM81d%2F978QaFoM7d2QAAAA%3D%3D

@jsquire do you know who owns Azure.Monitor.OpenTelemetry.AspNetCore?

[m-redding]

@pallavit I think Azure.Monitor.OpenTelemetry.AspNetCore is owned by @rajkumar-rangaraj or @TimothyMothra

[christothes]

The identity errors here are expected. This is because we only have tests with the filtered category of IdentityVM in Azure.Identity.Tests, but those tests run together with Azure.Identity.Broker.Tests.

@weshaggard Is there a better way to handle this filter which avoids this error?

[github-actions[bot]]

Hi @weshaggard, we deeply appreciate your input into this project. Regrettably, this issue has remained unresolved for over 2 years and inactive for 30 days, leading us to the decision to close it. We've implemented this policy to maintain the relevance of our issue queue and facilitate easier navigation for new contributors. If you still believe this topic requires attention, please feel free to create a new issue, referencing this one. Thank you for your understanding and ongoing support.

[github-actions[bot]]

Hi @weshaggard, we deeply appreciate your input into this project. Regrettably, this issue has remained unresolved for over 2 years and inactive for 30 days, leading us to the decision to close it. We've implemented this policy to maintain the relevance of our issue queue and facilitate easier navigation for new contributors. If you still believe this topic requires attention, please feel free to create a new issue, referencing this one. Thank you for your understanding and ongoing support.