More than one space in a string variable is reduced to zero spaces

[chrikoehn]

This code in a script node

string text;
text = "s          e";
print(text);
text = "Plenty of whitespace:'" + text + "'";
print(text);
print("s" + " " + " " + "e");

results in the following output (1&2 wrong, 3 correct):

se
Plenty of whitespace:'se'
s  e

This also affects speech synthesis nodes since using several spaces can result in something which cannot be spoken.

[timobaumann]

this is a bug in the lexer. however, I don't know anything about what's going on in there.

[timobaumann]

sequences of more than 1 NONNEWLINE_WHITE_SPACE_CHAR (tested space and tab) are reduced to nothing.

    @Test public void testStringParser() throws Exception {
        String testString = " s e ";
        Expression expr = Parser.parseExpression("\"" + testString + "\"", null);
        assertTrue(expr.evaluate() instanceof StringValue);
        assertEquals(expr.evaluate(), new StringValue(testString));
    }

[timobaumann]

line 362 in script.lex looks quite suspicious:

{NONNEWLINE_WHITE_SPACE_CHAR}+  { text(); }

although this is defined at the bottom and all the special stuff (e.g. string handling) is handled above.

[timobaumann]

see also: https://stackoverflow.com/questions/6736610/order-of-precedence-for-token-matching-in-flex

[alexanderkoller]

Great! Mention it in the CHANGELOG too?