search

nunit / nunit-console

NUnit Console runner and test engine
MIT License
214 stars 151 forks source link

\u001a test case fails on dotnet6 #1349

Open indy-singh opened 1 year ago

indy-singh commented 1 year ago 
[TestCase("\u001a")]
public void Then(string foo) => Assert.Pass();

The above testcase works fine on net472 + NUnit Console 3.16.3 (Release).

But it fails on dotnet6.0 + NUnit Console 3.16.3 (Release):-

With this exception:-

--XmlException
'', hexadecimal value 0x1A, is an invalid character. Line 1, position 1571.
   at System.Xml.XmlTextReaderImpl.Throw(Exception e)
   at System.Xml.XmlTextReaderImpl.ThrowInvalidChar(Char[] data, Int32 length, Int32 invCharPos)
   at System.Xml.XmlTextReaderImpl.ParseCDataOrComment(XmlNodeType type, Int32& outStartPos, Int32& outEndPos)
   at System.Xml.XmlTextReaderImpl.ParseCDataOrComment(XmlNodeType type)
   at System.Xml.XmlTextReaderImpl.ParseElementContent()
   at System.Xml.XmlLoader.LoadNode(Boolean skipOverWhitespace)
   at System.Xml.XmlLoader.LoadDocSequence(XmlDocument parentDoc)
   at System.Xml.XmlDocument.Load(XmlReader reader)
   at System.Xml.XmlDocument.LoadXml(String xml)
   at NUnit.XmlHelper.CreateXmlNode(String xml)
   at NUnit.Engine.Runners.WorkItemTracker.NUnit.Engine.ITestEventListener.OnTestEvent(String report)
   at NUnit.Engine.Runners.TestEventDispatcher.OnTestEvent(String report)
   at NUnit.Engine.Runners.MasterTestRunner.RunTests(ITestEventListener listener, TestFilter filter)
   at NUnit.Engine.Runners.MasterTestRunner.Run(ITestEventListener listener, TestFilter filter)
   at NUnit.ConsoleRunner.ConsoleRunner.RunTests(TestPackage package, TestFilter filter)
   at NUnit.ConsoleRunner.Program.Main(String[] args)

Cheers, Indy

indy-singh commented 1 year ago

Setup a minimal repo https://github.com/indy-singh/nunit_console_runner_issue_1349

ueli-werner commented 1 year ago

This is more likely a breaking change by Microsoft between Framework 4.7.2 and NET 6.

In my opinion this is not a bug in NUnit. The argument is a dotnet string and should be valid. Consider changing the argument type.

manfred-brands commented 1 year ago

The string is a valid .NET string, just not valid inside XML CDATA where characters are limited to: image

This excludes 0x1A.

Having said that, dotnet test has no problem with this However even though tests are shown in Visual Studio Test Explorer. image

They don't seem to run. I see error message: NUnit3TestExecutor discovered 0 of 0 NUnit test cases using Current Discovery mode, Explicit run

I don't know if the XML is NUnit specific or dictated by the test specification. If arguments are stored there, they must be stored in such a way that they result in valid XML. Maybe they need encoding.

andrewimcclement commented 8 months ago

Presumably there is a suitable escape mechanism we can use when creating the XML in the first place?

mikkelbu commented 8 months ago

I think the XML is NUnit specific and the framework have special code to handle this - see e.g. https://github.com/nunit/nunit/blob/68d994772d10de54ac960ea8379ee966a3293083/src/NUnitFramework/framework/XmlExtensions.cs#L76 - but apparent we also need some escaping in the engine.

CharliePoole commented 1 day ago

@indy-singh I think this may have been fixed along with #1469. Can you check against 3.18.2 or the latest 3.18.3 build on our MyGet feed?

indy-singh commented 3 hours ago

@CharliePoole

Using 

@echo off
choco install -y nunit-console-runner --version=3.18.2
choco install -y nunit-extension-vs-project-loader --version=3.9.0
dotnet build
nunit3-console.exe .\bin\Debug\net472\nunit_console_runner_issue_1349.dll
nunit3-console.exe .\bin\Debug\net6.0\nunit_console_runner_issue_1349.dll
pause

Output

Chocolatey v0.11.3
Installing the following packages:
nunit-console-runner
By installing, you accept licenses for the packages.
Progress: Downloading nunit-console-runner 3.18.2... 100%

nunit-console-runner v3.18.2 [Approved]
nunit-console-runner package files install completed. Performing other installation steps.
 ShimGen has successfully created a shim for nunit3-console.exe
 The install of nunit-console-runner was successful.
  Software install location not explicitly set, it could be in package or
  default install location of installer.

Chocolatey installed 1/1 packages.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
Chocolatey v0.11.3
Installing the following packages:
nunit-extension-vs-project-loader
By installing, you accept licenses for the packages.
nunit-extension-vs-project-loader v3.9.0 already installed.
 Use --force to reinstall, specify a version to install, or try upgrade.

Chocolatey installed 0/1 packages.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).

Warnings:
 - nunit-extension-vs-project-loader - nunit-extension-vs-project-loader v3.9.0 already installed.
 Use --force to reinstall, specify a version to install, or try upgrade.
  Determining projects to restore...
  All projects are up-to-date for restore.
  nunit_console_runner_issue_1349 -> C:\Users\REDACTED\Documents\Projects\nunit_console_runner_issue_1349\bin\Debug\net
  472\nunit_console_runner_issue_1349.dll
  nunit_console_runner_issue_1349 -> C:\Users\REDACTED\Documents\Projects\nunit_console_runner_issue_1349\bin\Debug\net
  6.0\nunit_console_runner_issue_1349.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:00.92

Workload updates are available. Run `dotnet workload list` for more information.
NUnit Console Runner 3.18.2 (Release)
Copyright (c) 2022 Charlie Poole, Rob Prouse
04 October 2024 14:40:03

Runtime Environment
   OS Version: Microsoft Windows NT 6.2.9200.0
   Runtime: .NET Framework CLR v4.0.30319.42000

Test Files
    .\bin\Debug\net472\nunit_console_runner_issue_1349.dll

Run Settings
    DisposeRunners: True
    WorkDirectory: C:\Users\REDACTED\Documents\Projects\nunit_console_runner_issue_1349
    ImageRuntimeVersion: 4.0.30319
    ImageTargetFrameworkName: .NETFramework,Version=v4.7.2
    ImageRequiresX86: False
    ImageRequiresDefaultAppDomainAssemblyResolver: False
    TargetRuntimeFramework: net-4.7.2
    NumberOfTestWorkers: 8

Test Run Summary
  Overall result: Passed
  Test Count: 1, Passed: 1, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
  Start time: 2024-10-04 13:40:03Z
    End time: 2024-10-04 13:40:04Z
    Duration: 0.768 seconds

Results (nunit3) saved as TestResult.xml
NUnit Console Runner 3.18.2 (Release)
Copyright (c) 2022 Charlie Poole, Rob Prouse
04 October 2024 14:40:04

Runtime Environment
   OS Version: Microsoft Windows NT 6.2.9200.0
   Runtime: .NET Framework CLR v4.0.30319.42000

Test Files
    .\bin\Debug\net6.0\nunit_console_runner_issue_1349.dll

System.Xml.XmlException : '→', hexadecimal value 0x1A, is an invalid character. Line 1, position 823.

--XmlException
'→', hexadecimal value 0x1A, is an invalid character. Line 1, position 823.
   at System.Xml.XmlTextReaderImpl.Throw(Exception e)
   at System.Xml.XmlTextReaderImpl.ThrowInvalidChar(Char[] data, Int32 length, Int32 invCharPos)
   at System.Xml.XmlTextReaderImpl.ParseCDataOrComment(XmlNodeType type, Int32& outStartPos, Int32& outEndPos)
   at System.Xml.XmlTextReaderImpl.ParseCDataOrComment(XmlNodeType type)
   at System.Xml.XmlTextReaderImpl.ParseElementContent()
   at System.Xml.XmlLoader.LoadNode(Boolean skipOverWhitespace)
   at System.Xml.XmlLoader.LoadDocSequence(XmlDocument parentDoc)
   at System.Xml.XmlDocument.Load(XmlReader reader)
   at System.Xml.XmlDocument.LoadXml(String xml)
   at NUnit.ConsoleRunner.TestEventHandler.OnTestEvent(String report)
   at NUnit.Engine.Runners.TestEventDispatcher.OnTestEvent(String report)
   at NUnit.Engine.Runners.MasterTestRunner.RunTests(ITestEventListener listener, TestFilter filter)
   at NUnit.Engine.Runners.MasterTestRunner.Run(ITestEventListener listener, TestFilter filter)
   at NUnit.ConsoleRunner.ConsoleRunner.RunTests(TestPackage package, TestFilter filter)
   at NUnit.ConsoleRunner.Program.Main(String[] args)

Press any key to continue . . .
CharliePoole commented 1 hour ago

So not fixed in 3.18.2. Can you try this...

choco install -y nunit-console-runner --version 3.18.3-dev00006 --source https://www.myget.org/F/nunit/api/v2