mono / monodevelop

MonoDevelop is a cross platform .NET IDE
http://www.monodevelop.com
2.85k stars 1.02k forks source link

Exception thrown when trying to run tests #3458

Open decriptor opened 6 years ago

decriptor commented 6 years ago

I get this exception when trying to run a single unit test. It would seem that there are more bugs in play here. It is a single [Test], but the output says that 3 tests were run:

Test Run Summary
  Overall result: Passed
  Test Count: 3, Passed: 3, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
  Start time: 2017-12-05 18:47:29Z
    End time: 2017-12-05 18:47:34Z
    Duration: 4.936 seconds

Results (nunit2) saved as /var/folders/6c/5qfjrpnx55x8jy0drk277vxr0000gn/T/tmp49d3bf3b.tmp

However, the test results pad shows zeros:

screenshot 2017-12-05 11 49 28

Here is the exception:

ERROR [2017-12-05 10:44:27Z]: Exception in test listener
System.ArgumentOutOfRangeException: startIndex cannot be larger than length of string.
Parameter name: startIndex
  at System.String.Substring (System.Int32 startIndex, System.Int32 length) [0x00087] in /Users/builder/data/lanes/5533/mono-mac-sdk/external/bockbuild/builds/mono-x64/mcs/class/referencesource/mscorlib/system/string.cs:1266 
  at System.String.Substring (System.Int32 startIndex) [0x00000] in /Users/builder/data/lanes/5533/mono-mac-sdk/external/bockbuild/builds/mono-x64/mcs/class/referencesource/mscorlib/system/string.cs:1232 
  at MonoDevelop.UnitTesting.NUnit.External.TcpTestListener.UpdateTestSuiteStatus (System.String name, System.Boolean isTest) [0x0000e] in /Users/builder/data/lanes/432/ef1df79a/source/monodevelop/main/src/addins/MonoDevelop.UnitTesting.NUnit/MonoDevelop.UnitTesting.NUnit/TcpTestListener.cs:136 
  at MonoDevelop.UnitTesting.NUnit.External.TcpTestListener+<>c__DisplayClass14_0.<.ctor>b__0 () [0x00163] in /Users/builder/data/lanes/432/ef1df79a/source/monodevelop/main/src/addins/MonoDevelop.UnitTesting.NUnit/MonoDevelop.UnitTesting.NUnit/TcpTestListener.cs:104 

VS bug #577083

decriptor commented 6 years ago

Ok, I think I found the repro and/or the cause. I had two tests with the same name, but not the same signature.

I'm getting mixed results, but it still highlights the issue

        [TestCase ("1", "2", TestName = "Missing")]
        [TestCase ("1", "2", TestName = "Empty")]
        public void Test1 (string one, string two)
        {
            Assert.AreEqual ("1", one);
            Assert.AreEqual ("2", two);
        }

        //[Repeat (40)]
        [Test]
        public void Test1 ()
        {
            Assert.AreEqual (1, 1);
        }