Current lexicon lookup implementation with POS tags is ambiguous.
The token "dell'X" is found in the lexicon because the word "dell'" is in the lexicon with POS tag "X". A solution is to change the code so that a # is always inserted between the word and the POS and that # and \ are escaped with a \ for both the POS and the word.
Current lexicon lookup implementation with POS tags is ambiguous. The token "dell'X" is found in the lexicon because the word "dell'" is in the lexicon with POS tag "X". A solution is to change the code so that a # is always inserted between the word and the POS and that # and \ are escaped with a \ for both the POS and the word.