JsonTextWriter produces invalid JSON when Unicode char \u201d present in string

[paulhickman-ec]

Expected behavior

Valid JSON to be produced with the character after testquote being ” (the Unicode right double quote character \u201d)

{"blah":"testquote”"}

Actual behavior

Invalid JSON is produced with the last character of the string being " (an ASCII quote \u0034)

{"blah":"testquote""}

Steps to reproduce

Put the following in a .NET 7 console app:

using Newtonsoft.Json;
using var s = new JsonTextWriter(Console.Out);
s.StringEscapeHandling = StringEscapeHandling.EscapeHtml;
s.WriteStartObject();
s.WritePropertyName("blah");
s.WriteValue("testquote”");
s.WriteEndObject();
Console.ReadLine();

Note: the character after testquote is \u201d not \u0034

If I use StringEscapeHandling.EscapeHtml or StringEscapeHandling.EscapeDefault it converts the quote to ".

If I use StringEscapeHandling.EscapeNonAscii I get the valid JSON output {"blah":"testquote\u201d"}

However, I can't make it produce JSON with the Unicode right quote as a literal.

Versions

Newtonsoft.Json 13.0.3 Visual Studio 17.7.7 .Net 7

[elgonzo]

Not a bug in Newtonsoft.Json. Newtonsoft.Json produces valid JSON and does not change the \u201d character to the \u0034 double quote.

What you see there is either an effect of the console doing the conversion if it is not using Unicode/UTF as a code page and/or the Console.Out TextWriter having a non-Unicode/non-UTF text encoding assigned, or perhaps a conversion happening by you perhaps copy'n'pasting the the console output into an editor or something.

I suggest you test in a more controlled manner eliminating any possible environmental factors that potentially can have an impact on how the text is being outputted/presented. In this case for example, instead of writing to Console.Out, write into a StringWriter. Then, after finishing writing the json data to the StringWriter, obtain the string from it and print out the ordinal value of the character (cast the char value to an int) at the string position of the right double quote \u201d to see that the \u201d character is not being substituted by Newtonsoft.Json.