goodmami / penman

PENMAN notation (e.g. AMR) in Python
https://penman.readthedocs.io/
MIT License
135 stars 26 forks source link

Allow non-breaking spaces in symbols? #99

Closed goodmami closed 2 years ago

goodmami commented 3 years ago

Consider this:

>>> penman.decode('(a/a :name (b / 1\u00a02\u00a03))')
Traceback (most recent call last):
[...]
penman.exceptions.DecodeError: 
  line 1
    (a/a :name (b / 1 2 3))
                      ^
DecodeError: Expected: ROLE

This example is adapted from https://github.com/jcyk/AMR-gs/issues/15.

Should non-breaking spaces not be broken during parsing? If so, what other spaces should be allowed, or rather, which should be disallowed in symbols?

goodmami commented 2 years ago

There are many breaking spaces (see https://en.wikipedia.org/wiki/Whitespace_character) and only a handful of non-breaking spaces. However, in a data format like PENMAN it seems reasonable to only allow standard ASCII spaces as breaking, primarily space, tab, line feed, and carriage return, but maybe also form feed and vertical tab. All others, including non-breaking space, ideographic space, zero-width word joiner, etc., are then treated as regular characters.