neuecc / Utf8Json

Definitely Fastest and Zero Allocation JSON Serializer for C#(NET, .NET Core, Unity, Xamarin).
MIT License
2.36k stars 267 forks source link

JsonSerializer.Deserialize reads outside buffer with Stream with malformed json #127

Open hpbieker opened 5 years ago

hpbieker commented 5 years ago
var stream1 = new MemoryStream(Encoding.UTF8.GetBytes("[1]"));
var result1 = JsonSerializer.Deserialize<int[]>(stream1);

var stream2 = new MemoryStream(Encoding.UTF8.GetBytes("["));
var result2 = JsonSerializer.Deserialize<int[]>(stream2);

result1 is now int[] { 1} as expected.

However, result2 is now also int[] { 1}, but I expected to get an exception because of invalid json.

The reason is that Deserialize<T>(Stream, IJsonFormatterResolver) will call MemoryPool.GetBuffer() which returns an unclean buffer. This buffer is then filled based on the stream in FillFromStream(). In our case, the second broken json is shorter than the first one, so our new buffer contains a mix of the new and the old json.

The buffer is then just passed on to the parser without any length field, so the whole buffer is potentially read if it contains garbage.

keithbeller commented 4 years ago

Is there a work around for this issue?

tomasfreund commented 4 years ago

Is there a work around for this issue?

Manually copy the content of the stream to a byte[] and deserialize the json from there

penguinawesome commented 4 years ago

Does this affect the JsonSerializer.NonGeneric.Deserialize ?

tomasfreund commented 4 years ago

Does this affect the JsonSerializer.NonGeneric.Deserialize ?

Yes, if you use the overload with Stream.

penguinawesome commented 4 years ago

Yes we do use the overload with stream since we are creating our own mediaformatter for asp.net web api. How about for serializing? JsonSerializer.NonGeneric.SerializeAsync ?