__eof__ is more likely to mask errors since hitting the end of the file is a normal occurrence. Ending early might result in the rest of the file being silently ignored and potentially lexing and parsing "successfully".
__jam__ is intended to only occur when the lexer can't actually process the file, and should usually only occur when there's a hole in the lexer, which is already a problem. __jam__ getting produced is usually a sign of an error, so this would be more appropriate.
That said, I think having a new one specifically for no production might be more appropriate. Rules wouldn't need a special Produces flag if there were a token specifically for that. But then there would be a null token, with all the fun that comes with null. Users might get nuisance warnings about not handling the error token.
I think I'll go with the __nothing__ token for now. If it turns out to be more trouble than it's worth, I can always take it out again later.
__eof__is more likely to mask errors since hitting the end of the file is a normal occurrence. Ending early might result in the rest of the file being silently ignored and potentially lexing and parsing "successfully".__jam__is intended to only occur when the lexer can't actually process the file, and should usually only occur when there's a hole in the lexer, which is already a problem.__jam__getting produced is usually a sign of an error, so this would be more appropriate.That said, I think having a new one specifically for no production might be more appropriate. Rules wouldn't need a special
Producesflag if there were a token specifically for that. But then there would be anulltoken, with all the fun that comes withnull. Users might get nuisance warnings about not handling the error token.I think I'll go with the
__nothing__token for now. If it turns out to be more trouble than it's worth, I can always take it out again later.