The text reader distinguishes between invalid input and incomplete input. If it encounters incomplete data, it tries to add more data to the buffer and try again. If it encounters invalid data, it surfaces the error to the user.
This PR adds a test utility called DataStraw--an io::Read implementation that only yields a single byte of input per call to read(). By doing this, all subslices of input are tested for correct incompleteness detection.
As you can imagine, this surfaced a number of bugs that needed to be fixed. I've fixed everything the tests surfaced in the Ion 1.1 text parser. In doing so I naturally also fixed the majority of bugs in the Ion 1.0 parser because they share lots of parsing logic. However, I have not set up the same tests for Ion 1.0 yet. I will do that in a follow-on PR.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
The text reader distinguishes between invalid input and incomplete input. If it encounters incomplete data, it tries to add more data to the buffer and try again. If it encounters invalid data, it surfaces the error to the user.
This PR adds a test utility called
DataStraw--anio::Readimplementation that only yields a single byte of input per call toread(). By doing this, all subslices of input are tested for correct incompleteness detection.As you can imagine, this surfaced a number of bugs that needed to be fixed. I've fixed everything the tests surfaced in the Ion 1.1 text parser. In doing so I naturally also fixed the majority of bugs in the Ion 1.0 parser because they share lots of parsing logic. However, I have not set up the same tests for Ion 1.0 yet. I will do that in a follow-on PR.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.