Invalid tokens generate an error without associated source information

[MichaelBurge]

Issue

When a token not mentioned in a brag grammar is given to the parser, the error does not include source information.

This can be reproduced by running racket b.rkt after creating two files a.rkt and b.rkt with the contents below.

Source code

a.rkt

#lang brag

example: TOKEN

b.rkt

#lang racket

(require "a.rkt")
(require brag/support)

(define tokens (list (position-token 'NONEXISTENT (position 74 5 12) (position 75 5 13))))
(define parse-tree (parse tokens))

Error message

Encountered unexpected token "NONEXISTENT" ('NONEXISTENT) while parsing #f [line=#f, column=#f, offset=#f]
  location...:
   #f

Requested fix

I propose rewording the error message as follows:

Token "NONEXISTENT" at [line=5, column=7, offset=74] is not in the parser's set of allowed tokens.

Valid token types are: [ <all possible token types> ]

[mbutterick]

(I didn’t rewrite the error message because it has side effects, but the source location will now appear in the message)