lezer-parser / lezer

Dev utils and issues for the Lezer core packages
33 stars 1 forks source link

generator creates invalid code #26

Closed BryanCrotaz closed 2 years ago

BryanCrotaz commented 2 years ago

I have written a simple grammar:

@top Program { expression }

expression { Name | Number | BinaryExpression }

BinaryExpression { "(" expression ("+" | "-") expression ")" }

@tokens {
  Name { @asciiLetter+ }
  Number { @digit+ }
}

then generated the parser;

lezer-generator ./test.grammar -o ./test.ts

and get

// This file was generated by lezer-generator. You probably shouldn't edit it.
import { LRParser } from '@lezer/lr';
export const parser = LRParser.deserialize({
  version: 14,
  states:
    "!QOQOPOOOQOPO'#C`OOOO'#Cb'#CbQOOOOOO]OPO,58zOQOPO1G.fOeOPO7+$QOOOO<<Gl<<Gl",
  stateData: 'j~OQQORQOVPO~OWTOXTO~OYVO~O',
  goto: 'fVPPPPWP]VQOPTQROQSPRUT',
  nodeNames: '⚠ Program Name Number BinaryExpression',
  maxTerm: 10,
  skippedNodes: [0],
  repeatNodeCount: 0,
  tokenData:
    '!a~RVxyhyzm{|r}!Ow!Q![|!c!}!U#T#o!U~mOV~~rOY~~wOW~~|OX~~!RPR~!Q![|~!ZQQ~!c!}!U#T#o!U',
  tokenizers: [0],
  topRules: { Program: [0, 1] },
  tokenPrec: 0,
});

Error: Property 'deserialize' does not exist on type 'typeof LRParser'.

I've searched the code base and deserialize doesn't appear in any of the parser source. Looks like the generator is out of date compared to the parser.

BryanCrotaz commented 2 years ago

Further investigation discovered your typescript definitions are missing the static deserialize:

// parse.d.ts

...
export declare class LRParser extends Parser {
  // add this line and create a type for spec as there isn't one defined
    static deserialize(spec: any): LRParser;

...
marijnh commented 2 years ago

That's valid code, as in it works, it's just not TypeScript code. You're not doing anything wrong except that you're running TypeScript on it.

BryanCrotaz commented 2 years ago

You publish typescript types, but they're wrong.

marijnh commented 2 years ago

No, they just don't show internals that the parser generator output uses.

BryanCrotaz commented 2 years ago

It's not internal if it ends up in the generated project file. That file is part of the app, so it will be type checked during the app build.

So that's public api not private.