Ichoran / kse

The Kerr Scala Extensions contain core functionality missing from the Scala standard library.
Other
28 stars 4 forks source link

Broken string parsing in Byte consumers #7

Open Ichoran opened 6 years ago

Ichoran commented 6 years ago

As reported by Andriy Plokhotnyuk, both the ByteBuffer and the ByteArrayInputStream (i.e. RecyclingParser) versions fail on the following input:

""""倒排索引(英语:Inverted index),也常被称为反向索引、置入档案或反向档案,是一种索引方法,被用来存储在全文搜索下某个单词在一个文档或者一组文档中的存储位置的映射。它是文档检索系统中最常用的数据结构。"""".getBytes

JsonRecyclingParser has the worse error in that it fails to even parse the string; JsonByteBufferParser parses successfully but encodes all the content as \uffXX where XX is the raw UTF-8 byte value.