parse results depend on the form of a terminal?

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Run bigloo3.2c
2. Go to lalr-scm-read-only/snowless
(load "lalr.scm")
3.

Run the following code:

(define (parse lexer::procedure errorp::procedure)
  (let* ()
    ((lalr-parser
      (output: grammar "grammar.scm")
      (out-table: "tables.scm")
      (DECLARE LONG ident)
      (top
       (DECLARE LONG ident) : $1
       )
      ) lexer errorp)))

(define lexer
  (let((tokens '(DECLARE
                 LONG
                 (ident . "i")
                 )))
    (lambda()
      (match-case tokens
                  ((?token . ?rest)
                   (set! tokens rest)
                   [print token]
                   token)
                  (else
                   '*eoi*)))))

(parse lexer print)

What is the expected output?
The sequence shall be parsed successfully.

What do you see instead?
Syntax error: invalid token: (ident . i)

What version of the product are you using?
svn version 35

On what operating system?
Linux x64

Please provide any additional information below.
If I use the symbol 'ident as the last token instead of the pair '(ident .
"i") the sequence parsed successfully.

Original issue reported on code.google.com by tsichev...@gmail.com on 5 Nov 2009 at 1:42

[GoogleCodeExporter]

The resulting state table is listed below. It looks Ok except that states 2 and 
4
seem to be unreachable.

State table
-----------

state 0

   *start* --> . top *eoi*    (rule 1)

   *default* : Error
   DECLARE : shift and goto state 1

state 1

   top --> DECLARE . LONG ident    (rule 2)

   *default* : Error
   LONG : shift and goto state 3

state 2

   *start* --> top . *eoi*    (rule 1)

   *default* : Error
   *eoi* : shift and goto state 4

state 3

   top --> DECLARE LONG . ident    (rule 2)

   *default* : Error
   ident : shift and goto state 5

state 4

   *start* --> top *eoi* .    (rule 1)

   *default* : reduce using rule 1
   *eoi* : Accept input

state 5

   top --> DECLARE LONG ident .    (rule 2)

   *default* : reduce using rule 2

Original comment by tsichev...@gmail.com on 5 Nov 2009 at 1:54

[GoogleCodeExporter]

From the documentation: 

http://code.google.com/p/lalr-scm/wiki/ParserDefinition

The token should either be a symbol or a record created by the function
make-lexical-token.

Original comment by schemeway on 5 Nov 2009 at 9:01

• Changed state: Invalid

[GoogleCodeExporter]

The problem was I used the the old token format (name/value pair) which is no 
more
suitable in version 2.4 of lalr generator, so this issue shall possible be 
closed.

Original comment by tsichev...@gmail.com on 5 Nov 2009 at 9:01