tree-sitter / tree-sitter-html

HTML grammar for Tree-sitter
MIT License
136 stars 72 forks source link

[wasm]build wasm and run in node env throw error #72

Closed wszgrcy closed 8 months ago

wszgrcy commented 11 months ago

npm i tree-sitter-html(0.20.0) npm i web-tree-sitter(0.20.8) build command npx tree-sitter build-wasm ./node_modules/tree-sitter-html build with tree-sitter-cli(0.20.8) and generate html.wasm(18kb) then I run with

  let wasm = readFileSync(path.join(__dirname, './html.wasm'));
  let grammar = await Parser.Language.load(new Uint8Array(wasm));
  const parser = new Parser();  
  parser.setLanguage(grammar);
  parser.setLogger((msg, data,type) => {
            console.log(msg, data,type);
        });
  let result = parser.parse(`<div>1234</div>`);
  console.log(result);

throw err

  at e.<computed> (/my/library/node_modules/web-tree-sitter/tree-sitter.js:1:16465)
        at wasm://wasm/00011d3a:wasm-function[14]:0xaa2
        at wasm://wasm/000b627a:wasm-function[234]:0x25842
        at Object.Module._ts_parser_parse_wasm (/my/library/node_modules/web-tree-sitter/tree-sitter.js:1:33493)
        at Parser.parse (/my/library/node_modules/web-tree-sitter/tree-sitter.js:1:53325)
        at new CssSelectorForTreeSitter (/my/library/src/selector/tree-sitter/selector.ts:25:29)
        at createCssSelectorForTreeSitter (/my/library/src/selector/tree-sitter/selector.ts:14:12)
        at async UserContext.<anonymous> (/my/library/src/selector/tree-sitter/selector.spec.ts:29:21)

if I run with 1234 then It will worked logger

new_parse false undefined
process version:0, version_count:1, state:1, row:0, col:0 false undefined
lex_external state:2, row:0, column:0 false undefined
consume character:'<' true undefined
lex_internal state:56, row:0, column:0 false undefined
consume character:'<' true undefined
lexed_lookahead sym:<, size:2 false undefined
shift state:52 false undefined
process version:0, version_count:1, state:52, row:0, col:2 false undefined
lex_external state:5, row:0, column:2 false undefined
consume character:'d' true undefined
consume character:'i' true undefined
consume character:'v' true undefined
wszgrcy commented 11 months ago

I git clone this repo and build wasm and run with same error. Is it necessary to provide test cases for the wasm version?

amaanq commented 9 months ago

your example worked just fine for me, did you build the wasm file from HEAD? or what version? (hopefully post C++ scanner rewrite into C)

wszgrcy commented 8 months ago

npx tree-sitter build-wasm ./node_modules/tree-sitter-html

I use commit 5760f521fac79263de4ca71a590ef516594c9f37(The latest code at the time of reply),But there is still an error my pc environment: Linux pc-MS-7C94 6.5.0-18-generic #18~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Feb 7 11:40:03 UTC 2 x86_64 x86_64 x86_64 GNU/Linux gcc --version g++ --version

gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0

Can you provide your build environment so that I can use Docker to build it

Or I need to provide those key information to help troubleshoot

wszgrcy commented 8 months ago

your example worked just fine for me, did you build the wasm file from HEAD? or what version? (hopefully post C++ scanner rewrite into C)

I fork this repo and add github action,Still failing repo https://github.com/wszgrcy/tree-sitter-html action https://github.com/wszgrcy/tree-sitter-html/actions/runs/7991036597/job/21821254491

amaanq commented 8 months ago

ah okay, npm is on 0.20.8 which is too old, we will have a release today that should fix this. the issue is we reworked what symbols in scanners are allowed in wasm, but this hasn't been updated in the npm release yet (i was using a build of tree-sitter.js from master, sorry). There is a pre-release of 0.21.0 on npm anyways that could work as well fyi

Thanks for providing debug info! made my life easier

wszgrcy commented 8 months ago

ah okay, npm is on 0.20.8 which is too old, we will have a release today that should fix this. the issue is we reworked what symbols in scanners are allowed in wasm, but this hasn't been updated in the npm release yet (i was using a build of tree-sitter.js from master, sorry). There is a pre-release of 0.21.0 on npm anyways that could work as well fyi

Thanks for providing debug info! made my life easier

Thank you very much for your help

amaanq commented 8 months ago

0.21.0 is now released, that should work for you