I suspect that the $token->value values are sometimes (or perhaps always) being stored in an escaped form (e.g. when parsing Turtle unicode escapes like \u0063). This needs to be fixed, so that the stored value is always the actual unicode string intended. If this requires changes, then the Turtle and SPARQL serializers need to be verified as re-encoding where necessary.
I suspect that the
$token->value
values are sometimes (or perhaps always) being stored in an escaped form (e.g. when parsing Turtle unicode escapes like\u0063
). This needs to be fixed, so that the stored value is always the actual unicode string intended. If this requires changes, then the Turtle and SPARQL serializers need to be verified as re-encoding where necessary.