Extension host crashes on `unhandledRejection`

[Veetaha]

This issue is mostly FYI vscode-tree-sitter crashes the extension host process in case of any unhandled promise rejections. Unfortunately, you are not able to do much on the extension side. Most likely the culprit is your web-tree-sitter dependency. The related issue is https://github.com/tree-sitter/tree-sitter/issues/571#issue-577509898.

At the end of the day vscode-tree-sitter damages the extension host process when this happens and all other extensions altogether.

The initial issue was reported in https://github.com/rust-analyzer/rust-analyzer/issues/3515#issuecomment-596207493 to the unrelated rust-analyzer repo.

[georgewfraser]

There's been a lot of mysterious crashes on windows coming from the compiled WASM parsers for specific languages. I recently published a new version where I updated to the latest versions of all deps, rebuilt the WASM files, and removed verilog, which seemed to be causing problems. Hopefully that has fixed it...

[Veetaha]

Unfortunately, updating the dependencies doesn't guarantee that their wasm glue code no longer crashes the process on unhandledRejection. Anyway, to be 99% sure you just test-run vscode-tree-sitter with the following line of code inserted anywhere in the extension code (though probably after all the wasm modules are loaded):

Promise.reject(0);

[qarlosalberto]

Would be possible to use the verilog WASM for Linux? Or is there problems as in Windows?

[MewX]

Having the same issue (ArchLinux with vscode v1.50.0-insider):

localProcessExtensionHost.ts:253 /home/a/.vscode-insiders/extensions/georgewfraser.vscode-tree-sitter-0.1.25/nodemodules/web-tree-sitter/tree-sitter.js:1(function (exports, require, module, filename, dirname) { var Module=void 0!==Module?Module:{};!function(e,t){"function"==typeof define&&define.amd?define([],t):"object"==typeof exports?module.exports=t():window.TreeSitter=t()}(0,function(){var e,t={};for(e in Module)Module.hasOwnProperty(e)&&(t[e]=Module[e]);var r,n,o=[],s=function(e,t){throw t},=!1,a=!1;="object"==typeof window,a="function"==typeof importScripts,r="object"==typeof process&&"object"==typeof process.versions&&"string"==typeof process.versions.node,n=!&&!r&&!a;var u,i,l,d,c="";r?(c=a?require("path").dirname(c)+"/":__dirname+"/",u=function(e,t){return l||(l=require("fs")),d||(d=require("path")),e=d.normalize(e),l.readFileSync(e,t?null:"utf8")},i=function(e){var t=u(e,!0);return t.buffer||(t=new Uint8Array(t)),q(t.buffer),t},process.argv.length>1&&process.argv[1].replace(/\/g,"/"),o=process.argv.slice(2),"undefin[RuntimeError: abort(undefined). Build with -s ASSERTIONS=1 for more info. at process.de (/home/a/.vscode-insiders/extensions/georgewfraser.vscode-tree-sitter-0.1.25/node_modules/web-tree-sitter/tree-sitter.js:1:10170) at process.emit (events.js:228:7) at processPromiseRejections (internal/process/promises.js:201:33) at processTicksAndRejections (internal/process/task_queues.js:95:32)]

After several settings changes, keybindings changes, extension changes... It no longer crashed the extension manager...

[bugproof]

same issue

c:\Users\Konrad.vscode\extensions\georgewfraser.vscode-tree-sitter-0.1.25\nodemodules\web-tree-sitter\tree-sitter.js:1 (function (exports, require, module, filename, dirname) { var Module=void 0!==Module?Module:{};!function(e,t){"function"==typeof define&&define.amd?define([],t):"object"==typeof exports?module.exports=t():window.TreeSitter=t()}(0,function(){var e,t={};for(e in Module)Module.hasOwnProperty(e)&&(t[e]=Module[e]);var r,n,o=[],s=function(e,t){throw t},=!1,a=!1;="object"==typeof window,a="function"==typeof importScripts,r="object"==typeof process&&"object"==typeof process.versions&&"string"==typeof process.versions.node,n=!&&!r&&!a;var u,i,l,d,c="";r?(c=a?require("path").dirname(c)+"/":__dirname+"/",u=function(e,t){return l||(l=require("fs")),d||(d=require("path")),e=d.normalize(e),l.readFileSync(e,t?null:"utf8")},i=function(e){var t=u(e,!0);return t.buffer||(t=new Uint8Array(t)),q(t.buffer),t},process.argv.length>1&&process.argv[1].replace(/\/g,"/"),o=process.argv.slice(2),"undefined"!

[Error: EBUSY: resource busy or locked, lstat 'C:\DumpStack.log.tmp' Emitted 'error' event on FSWatcher instance at: at FSWatcher._handleError (c:\Users\Konrad.vscode\extensions\ms-vscode.cmake-tools-1.4.2\dist\main.js:7789:10) at ReaddirpStream.NodeFsHandler._boundHandleError (c:\Users\Konrad.vscode\extensions\ms-vscode.cmake-tools-1.4.2\dist\main.js:25690:43) at ReaddirpStream.emit (events.js:223:5) at emitErrorNT (internal/streams/destroy.js:92:8) at emitErrorAndCloseNT (internal/streams/destroy.js:60:3) at processTicksAndRejections (internal/process/task_queues.js:81:21)] { errno: -4082, code: 'EBUSY', syscall: 'lstat', path: 'C:\DumpStack.log.tmp' }

[ghost]

@georgewfraser Can all tree-sitter-* deps be updated? Currently the extension is using 0.16 which is broken:

• https://github.com/tree-sitter/tree-sitter/issues/571
• https://github.com/tree-sitter/tree-sitter/pull/822

[sogaiu]

@SNDST00M May be you know this already, but FWIW, there's this text at the top of the README:

Tree Sitter for VSCode [Deprecated]

I don't know what that means in this case though.

[ghost]

I'm fully aware but there is vast demand for Tree-sitter so a critical fix should be on the cards

https://github.com/microsoft/vscode/issues/50140

[jeff-hykin]

I have what is essentially a non-depecated version of this repo (I worked with @georgewfraser). I can give you (@SNDST00M) access to the mirror so changes can be made in the meantime.

Once/if George is back online we can merge the changes back into this repo since he really did most of the heavy lifting.

[sogaiu]

I didn't quite get it all to work (see development console for errors), but FWIW, here's some work on package.json:

{
    "name": "vscode-tree-sitter",
    "displayName": "Tree Sitter [Deprecated]",
    "description": "Accurate syntax coloring with tree-sitter",
    "icon": "tree-sitter-small.png",
    "version": "0.1.26",
    "preview": true,
    "publisher": "georgewfraser",
    "repository": {
        "type": "git",
        "url": "https://github.com/georgewfraser/vscode-tree-sitter"
    },
    "license": "MIT",
    "extensionKind": [
        "ui"
    ],
    "engines": {
        "vscode": "^1.45.0"
    },
    "categories": [
        "Programming Languages",
        "Themes",
        "Other"
    ],
    "activationEvents": [
        "onLanguage:go",
        "onLanguage:cpp",
        "onLanguage:rust",
        "onLanguage:ruby",
        "onLanguage:typescript",
        "onLanguage:javascript"
    ],
    "main": "./out/extension.js",
    "contributes": {
        "grammars": [
            {
                "language": "go",
                "scopeName": "source.go",
                "path": "./textmate/go.tmLanguage.json"
            },
            {
                "language": "cpp",
                "scopeName": "source.cpp",
                "path": "./textmate/cpp.tmLanguage.json"
            },
            {
                "language": "ruby",
                "scopeName": "source.ruby",
                "path": "./textmate/ruby.tmLanguage.json"
            },
            {
                "language": "rust",
                "scopeName": "source.rust",
                "path": "./textmate/rust.tmLanguage.json"
            },
            {
                "language": "typescript",
                "scopeName": "source.ts",
                "path": "./textmate/typescript.tmLanguage.json"
            },
            {
                "language": "javascript",
                "scopeName": "source.ts",
                "path": "./textmate/typescript.tmLanguage.json"
            }
        ]
    },
    "scripts": {
        "vscode:prepublish": "npm run compile",
        "compile": "npx tsc -p ./",
        "watch": "npx tsc -watch -p ./",
        "test": "npm run compile && node ./out/test",
        "benchmark": "npm run compile && node ./out/benchmark",
        "debug": "npm run compile && node --nolazy --inspect-brk=9229 ./out/test",
        "build": "npx vsce package -o build.vsix",
        "publish": "npx vsce publish patch"
    },
    "devDependencies": {
        "@types/mocha": "^5.2.7",
        "@types/node": "^8.10.25",
        "@types/vscode": "^1.45.0",
        "tree-sitter-cli": "^0.19.5",
        "tree-sitter-cpp": "^0.19.0",
        "tree-sitter-go": "^0.19.0",
        "tree-sitter-javascript": "^0.19.0",
        "tree-sitter-ruby": "^0.19.0",
        "tree-sitter-rust": "^0.19.0",
        "tree-sitter-typescript": "^0.19.0",
        "tslint": "^6.0.0",
        "typescript": "^3.8.2",
        "vscode-test": "^1.3.0"
    },
    "dependencies": {
        "jsonc-parser": "^2.1.0",
        "tar": ">=4.4.2",
        "web-tree-sitter": "^0.19.0"
    }
}

[jeff-hykin]

Sorry about that previous link @SNDST00M @sogaiu, I had accidentally overwritten master with an old/very-broken WIP local branch and haven't checked that repo in a while. I fixed the branches, and master is now the stable version up to date with this repo.

@sogaiu I updated the npm dependencies which fixed many of the initial issues. Then I tested building and installing. Since node-gyp is used, and it can be a pain to setup, I created an environment script that installs the C++ build tools and python version automatically so that everything works out of the box.

Let me know if you're able to follow the setup instructions and run project compile. It should generate a build.vsix, and then you can just right-click install that file into VS Code.

[ghost]

I'll take a look tomorrow at porting Bash. I find VSC's Bash support to be very lacking.

[jeff-hykin]

@SNDST00M I agree that it's lacking, I created an improved textmate syntax for it here with a few thousand downloads. Its still got problems, but it looks a lot better

[sogaiu]

@jeff-hykin Thanks for the detailed instructions.

TLDR: tried some things and currently facing an error

Details:

I tried to follow the setup instructions to install nix, cloned your repository, and ran project compile.

This did produce a build.vsix.

I installed the .vsix in version 1.58.0 of some flavor of VSCode ( https://archlinux.org/packages/community/x86_64/code/ ).

I get the following error in the developer console:

workbench.desktop.main.js:2350 Activating extension 'jeff-hykin.experimental-tree-sitter' failed: Cannot find module '/home/user/.vscode-oss/extensions/jeff-hykin.experimental-tree-sitter-0.2.1/out/extension.js'
Require stack:
- /usr/lib/code/out/vs/loader.js
- /usr/lib/code/out/bootstrap-amd.js
- /usr/lib/code/out/bootstrap-fork.js.

I looked inside ~/.vscode-oss/extensions/jeff-hykin.experimental-tree-sitter-0.2.1 but saw no out directory. Nor does there appear to be one in the cloned repository directory where the build presumably took place.

I tried removing "noEmit: true" from tsconfig.json ( https://github.com/jeff-hykin/experimental-tree-sitter/blob/master/tsconfig.json#L16 ) and reran project compile.

This resulted in out being created in the cloned project repository directory as well as ending up inside build.vsix.

I now get different error(s) in the development console (seen after opening a .ts file): vscode.log

[jeff-hykin]

@sogaiu thanks for the detailed feedback. Looks like I screwed up when I changed that tsconfig. I was getting a false positive when testing changed because I had already generated an out folder.

I definitely generated several working build.vsix files, but the bad news is now I am essentially stuck at your same issue. I went back through all the commits I made, generating a build.vsix at each one, and so far all of them crash the entire extension host without leaving a message. I have no idea what changed.

I rolled back some code-organization changes on master (like the one that added emit: false), and put all the most recent changes such as improving the compile script (deletes out folder and build.vsix to prevent false positives now) on a wip branch.

Now that I think about it, when I was exploring previous commits it also reverts the project compile command, so I probably need to test older commits while using the newer command. I've got some other things to take care of though so it might be a while before I'm able to pick back up on this topic.

[jeff-hykin]

One other thing to note, there seems to be another extension (I believe it's called syntax highlighter) that uses the tree sitter, but seems to be actively maintained. It might be worthwhile to switch to focusing on that repo.

If you copy the commands, resources, and settings folder into a fork of that actively maintained repo the commands/start will most likely still work and autoinstall the node-gyp dependencies of that repo. The compile command might be different, but I'm sure they'd have an npm script for it

[ghost]

This is the extension you might be thinking of:

https://marketplace.visualstudio.com/items?itemName=jeff-hykin.experimental-tree-sitter

Oh you're the author! (Dev interview moment)

[sogaiu]

May be it's this one?

https://github.com/EvgeniyPeshkov/syntax-highlighter

I recognize the author from this discussion.

[ghost]

Yeah looks good to me. It even uses LSP to support existing Textmate themes.

[ghost]

Could you post it in microsoft/vscode#50140 so that Microsoft staff can resolve the issue?

[jeff-hykin]

May be it's this one?

https://github.com/EvgeniyPeshkov/syntax-highlighter

I recognize the author from this discussion.

Yup that's the one, I didn't realize he had kept maintaining it all this time. Looks like he's already using semantic highlighting which could be a much better replacement than trying to use the LSP!

[sogaiu]

Thanks for the confirmation and identifying this extension!