Flaky unit test DisposeMethodsShouldCallBaseClassDisposeTests.ErrorCase_NoDiagnostic

[mavasani]

This test is occasionally failing in our signed/CI builds. Seems like compiler is sometimes reporting an extra CS error. We may want to consider updating/removing this test from this repo to workaround this (while also trying to identify a repro case for compiler issue).

Failing test: https://github.com/dotnet/roslyn-analyzers/blob/a23fde8d03fcec0c8e5beb6b1f3882048ab8435b/src/NetAnalyzers/UnitTests/Microsoft.NetCore.Analyzers/Runtime/DisposeMethodsShouldCallBaseClassDisposeTests.cs#L1083-L1117

Test failure log:

Microsoft.CodeAnalysis.Testing.Verifiers.EqualWithMessage
Exception : Context: Diagnostics of test state
Mismatch between number of diagnostics returned, expected "7" actual "8"

Diagnostics:
// /0/Test0.cs(2,29): error CS0246: El nombre del tipo o del espacio de nombres 'IComparable<>' no se encontró (¿falta una directiva using o una referencia de ensamblado?)
DiagnosticResult.CompilerError("CS0246").WithSpan(2, 29, 2, 43).WithArguments("IComparable<>"),
// /0/Test0.cs(3,16): error CS0246: El nombre del tipo o del espacio de nombres 'IComparable<>' no se encontró (¿falta una directiva using o una referencia de ensamblado?)
DiagnosticResult.CompilerError("CS0246").WithSpan(3, 16, 3, 30).WithArguments("IComparable<>"),
// /0/Test0.cs(3,16): error CS0246: El nombre del tipo o del espacio de nombres 'IComparable<>' no se encontró (¿falta una directiva using o una referencia de ensamblado?)
DiagnosticResult.CompilerError("CS0246").WithSpan(3, 16, 3, 30).WithArguments("IComparable<>"),
// /0/Test0.cs(10,22): error CS1061: "T" no contiene una definición para "CompareTo" ni un método de extensión accesible "CompareTo" que acepte un primer argumento del tipo "T" (¿falta alguna directiva using o una referencia de ensamblado?)
DiagnosticResult.CompilerError("CS1061").WithSpan(10, 22, 10, 31).WithArguments("T", "CompareTo"),
// /0/Test0.cs(13,26): error CS0115: 'BaseClass<T>.Equals(object)': no se encontró ningún miembro adecuado para invalidar
DiagnosticResult.CompilerError("CS0115").WithSpan(13, 26, 13, 32).WithArguments("BaseClass<T>.Equals(object)"),
// /0/Test0.cs(23,25): error CS0115: 'BaseClass<T>.GetHashCode()': no se encontró ningún miembro adecuado para invalidar
DiagnosticResult.CompilerError("CS0115").WithSpan(23, 25, 23, 36).WithArguments("BaseClass<T>.GetHashCode()"),
// /0/Test0.cs(26,14): error CS0314: El tipo 'T' no se puede usar como parámetro de tipo 'T' en el tipo o método genérico 'BaseClass<T>'. No hay conversión boxing ni conversión de parámetro de tipo de 'T' a 'IComparable<T>'.
DiagnosticResult.CompilerError("CS0314").WithSpan(26, 14, 26, 26).WithArguments("BaseClass<T>", "IComparable<T>", "T", "T"),
// /0/Test0.cs(27,15): error CS0246: El nombre del tipo o del espacio de nombres 'IComparable<>' no se encontró (¿falta una directiva using o una referencia de ensamblado?)
DiagnosticResult.CompilerError("CS0246").WithSpan(27, 15, 27, 29).WithArguments("IComparable<>"),

Assert.Equal() Failure\r\nExpected: 7\r\nActual:   8

[Youssef1313]

This appears to be the same issue as https://github.com/dotnet/roslyn/issues/45877

[mavasani]

    Microsoft.NetCore.Analyzers.Runtime.UnitTests.DisposeMethodsShouldCallBaseClassDisposeTests.ErrorCase_NoDiagnostic [FAIL]
      Microsoft.CodeAnalysis.Testing.Verifiers.EqualWithMessageException : Context: Diagnostics of test state
      Mismatch between number of diagnostics returned, expected "7" actual "8"

      Diagnostics:
      // /0/Test0.cs(2,29): error CS0246: The type or namespace name 'IComparable<>' could not be found (are you missing a using directive or an assembly reference?)
      DiagnosticResult.CompilerError("CS0246").WithSpan(2, 29, 2, 43).WithArguments("IComparable<>"),
      // /0/Test0.cs(3,16): error CS0246: The type or namespace name 'IComparable<>' could not be found (are you missing a using directive or an assembly reference?)
      DiagnosticResult.CompilerError("CS0246").WithSpan(3, 16, 3, 30).WithArguments("IComparable<>"),
      // /0/Test0.cs(3,16): error CS0246: The type or namespace name 'IComparable<>' could not be found (are you missing a using directive or an assembly reference?)
      DiagnosticResult.CompilerError("CS0246").WithSpan(3, 16, 3, 30).WithArguments("IComparable<>"),
      // /0/Test0.cs(10,22): error CS1061: 'T' does not contain a definition for 'CompareTo' and no accessible extension method 'CompareTo' accepting a first argument of type 'T' could be found (are you missing a using directive or an assembly reference?)
      DiagnosticResult.CompilerError("CS1061").WithSpan(10, 22, 10, 31).WithArguments("T", "CompareTo"),
      // /0/Test0.cs(13,26): error CS0115: 'BaseClass<T>.Equals(object)': no suitable method found to override
      DiagnosticResult.CompilerError("CS0115").WithSpan(13, 26, 13, 32).WithArguments("BaseClass<T>.Equals(object)"),
      // /0/Test0.cs(23,25): error CS0115: 'BaseClass<T>.GetHashCode()': no suitable method found to override
      DiagnosticResult.CompilerError("CS0115").WithSpan(23, 25, 23, 36).WithArguments("BaseClass<T>.GetHashCode()"),
      // /0/Test0.cs(26,14): error CS0314: The type 'T' cannot be used as type parameter 'T' in the generic type or method 'BaseClass<T>'. There is no boxing conversion or type parameter conversion from 'T' to 'IComparable<T>'.
      DiagnosticResult.CompilerError("CS0314").WithSpan(26, 14, 26, 26).WithArguments("BaseClass<T>", "IComparable<T>", "T", "T"),
      // /0/Test0.cs(27,15): error CS0246: The type or namespace name 'IComparable<>' could not be found (are you missing a using directive or an assembly reference?)
      DiagnosticResult.CompilerError("CS0246").WithSpan(27, 15, 27, 29).WithArguments("IComparable<>"),

      Assert.Equal() Failure
      Expected: 7
      Actual:   8

[mavasani]

@Youssef1313 I think it seems best to skip this test with this issue and avoid CI flakiness.

[sharwell]

How often is this failing?

[Youssef1313]

@sharwell Not much frequent. I hit it 3 or 4 times at max in my PRs.

[sharwell]

I was originally against disabling this test, but now I've come all the way around the other way. We can delete this test altogether:

• It links to #1671, but this test is not related to that issue
• This test is in a class related to IDisposable analysis, but there is no Dispose method and no use of IDisposable

The test serves no purpose so I'll send a PR to delete it.

[Youssef1313]

@sharwell The case should still be forwarded to the compiler team (if it's not already a dupe of dotnet/roslyn#45877)