Closed goodmami closed 5 years ago
I think the result of a discussion for this was that it's not ideal to have the non-breaking space there, since usually spaces in predicates would get a +
to delimit the sides, but in pydelphin I will stop breaking on non-breaking spaces anyway.
The preprocessor now inserts an underscore rather than a non-breaking space for 50 000, so it shouldn't cause trouble for other tools that may not be as robust as pydelphin is now.
JHU profile item # 3035631
edit: example redacted because it came from a test corpus...
The
50 000
is (correctly) tokenized as a single number, but the predicate symbol includes the space directly:In general I think the correct thing to do is escape the space:
_50\ 000/nn_u_unknown
, but it's actually a no-break space character U+00A0 and not the regular space U+0020, so maybe my processor (pyDelphin, in this case) should um, not break at the no-break space?