how i can add list of position-token support for peg

[wwall]

Hello I have lexer, and function which transform input stream to vector of position-token elements (lexer construct lexer-src-pos from parser-tools/lex )

i have macros do-token (it's copy of do-literal in peg.rkt), which recognize token

(define-for-syntax (do-token string answer) (with-syntax ((string string) (answer answer))

`(let ((ls (quote (list (syntax->datum #'string)))))

    (lambda (input column)
      (let loop ((current column)
                 (letters ls))
        (log 3 "Letters ~a\n" letters)
        (if (null? letters)
            (begin
              (log 3 "Continuing to the next part after matching '~a'\n" string)
              ((answer string) input current))
            ;; (list string current)
            (begin
              (log 4 "Matching '~a' to input '~a' at column ~a\n" (car letters)
                   (input current)
                   current)
              (let ((next (token-name (position-token-token (input current)))))
                (cond
                  ((eq? next end-of-input) #f)
                  ((equal? next (car letters))
                   (loop (add1 current) (cdr letters)))
                  (else #f))))))))))

but i can't understand how add support of it to macros peg at the end i want use peg parser for list of token. for example (define-parser lxmComma (token 'lxmComma)) and take to input (list (position-token (token 'lxmComma ",") ... ...)

[kazzmir]

You want to make the peg invoke 'do-token'? I think you can do it by adding a case do 'translate-choice'. For instance, the predicate pattern matches on the keyword 'predicate'.

((predicate what) (store (do-predicate #'what #'rest)))

In your case it would be something like

((token what) (store (do-token #'what #'rest))

And you use (token ...stuff...) in the grammar.

(peg (start x) (grammar (x ((token z) z)))

Sorry if I'm misunderstanding you, its been a long time since I looked at the peg code.