Closed goodmami closed 2 years ago
There are many breaking spaces (see https://en.wikipedia.org/wiki/Whitespace_character) and only a handful of non-breaking spaces. However, in a data format like PENMAN it seems reasonable to only allow standard ASCII spaces as breaking, primarily space, tab, line feed, and carriage return, but maybe also form feed and vertical tab. All others, including non-breaking space, ideographic space, zero-width word joiner, etc., are then treated as regular characters.
Consider this:
This example is adapted from https://github.com/jcyk/AMR-gs/issues/15.
Should non-breaking spaces not be broken during parsing? If so, what other spaces should be allowed, or rather, which should be disallowed in symbols?