Any method to integrate a customized lexer in your library?

[asmwarrior]

Hi, as we know, the PEG grammar is text(character) based, which means we write lexer and parser grammar together. But in my application, I have my own lexer(mainly the C-preprocessor) implemented myself, so my question is: can the library support parsing rules for custom lexer.

For example, the lexer has some kinds of std::vector<Token> supplied, the Token class may have definition like:

class Token
{
    TokenKind kind;
    std::string lexeme;
}

Any idea or suggestions? I mean it could be a generalized peg grammar?

Thanks.

[yhirose]

@asmwarrior, thanks for your question. In short, it's difficult to do it...

Here is the interface of Ope which is a parser combinator.

  size_t parse_core(const char *s, size_t n, SemanticValues & /*vs*/,
                    Context &c, std::any & /*dt*/) const override {

As you can see, it takes a char pointer. If you can change it to take a list of tokens (and change all the related code for it), it might be doable. But it must be better and easier to write such a parser from scratch instead of changing cpp-peglib. And we can't call it a PEG parser. Hope it helps.

[asmwarrior]

OK, I see. Thanks for the reply.