Skip RS CTRL-CHAR to support JSON Text Sequences

[quaff]

Currently jackson supports reading newline-delimited json such as JSON Lines and NDJSON, see https://github.com/FasterXML/jackson-databind/issues/1304. There is a proposed standard RFC7464 call JSON Text Sequences, it's similar to newline-delimited json by add a leading RS CTRL-CHAR which is not accepted by jackson.

com.fasterxml.jackson.core.JsonParseException: Illegal character ((CTRL-CHAR, code 30)): only regular white space (\r, \n, \t) is allowed between tokens
 at [Source: (String)"{"name":"name1"}
{"name":"name2"}
"; line: 1, column: 2]
    at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1851)
    at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:707)
    at com.fasterxml.jackson.core.base.ParserMinimalBase._throwInvalidSpace(ParserMinimalBase.java:685)
    at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._skipWSOrEnd(ReaderBasedJsonParser.java:2397)
    at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:672)
    at com.fasterxml.jackson.databind.ObjectReader.readValues(ObjectReader.java:1898)

It would be nice if jackson simply treat RS as "\t" be default or JsonReadFeature.ALLOW_UNESCAPED_CONTROL_CHARS enabled. here is test case:

import static org.junit.Assert.assertEquals;

import java.util.ArrayList;
import java.util.List;

import org.junit.Test;

import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.MappingIterator;
import com.fasterxml.jackson.databind.ObjectMapper;

public class JSONTextSequencesTest {

    @Test
    public void test() throws Exception {
        String json = "\u001E{\"name\":\"name1\"}\n\u001E{\"name\":\"name2\"}\n";
        ObjectMapper om = new ObjectMapper();
        MappingIterator<JsonNode> it = om.readerFor(JsonNode.class).readValues(json);
        List<String> names = new ArrayList<>();
        while (it.hasNext())
            names.add(it.next().get("name").asText());
        assertEquals(2, names.size());
        assertEquals("name1", names.get(0));
        assertEquals("name2", names.get(1));
    }

}

[cowtowncoder]

It may make sense to support RS as alternate white-space, but given that it is not allowed by JSON specification itself, allowing it should not be default behavior. But one thing that perhaps could be improved would be fail message which could suggest enabling feature if 0x1E is encountered.

ALLOW_UNQUOTED_CONTROL_CHARS is bit tricky too, as its definition refers to String values, not white-space. I am not sure that should be changed either.

Which would leave the option of a new feature or two, relating to allowed white-space:

1. Option specifically allowing RS character
2. Option allowing all unquoted CTRL characters

One possible concern I have is performance, as skipping of white-space is quite heavily optimized, but without using lookup tables. So addition of a new code may have measurable effect.

[quaff]

I vote for option 2 if performance is not an issue.

[Creaturism]

Wondering if something like this is how a feature allowing the RS char specifically would be done

[KennyxLan]

Is this issue solved?

[pjfanning]

Is this issue solved?

Not done. This is a volunteer project. Would you like to volunteer?