[Feature Request] Parse files on disk

[RedCMD]

Currently this extension can only parse documents that are open in vscode and have a correctly assigned languageId

I think it would be nice to be able to parse any file from disk with a manually assigned language Would be very useful with things like the DefinitionProvider, which requires checking the contents of other files for the correct info

parseFile(uri, languageId)

[pokey]

@RedCMD I'm tempted to say that's out of scope for this extension, but it would be good to understand your use case a bit better. What are you hoping to use this extension for?

[RedCMD]

Making a extension for vscodes json based textmate I have syntax highlighting using textmate and a 'language server' for semantic highlighting and ctrl+click definitions etc I made a basic parser, but didn't like it very much so I wanted to use tree-sitter but couldn't get (node) web-tree-sitter working at all (which seems common) found this extension (seeing it been highly maintained) and currently using it to load tree-sitter and handle text edits seamlessly

[pokey]

Interesting. What are you planning to use the custom wasm support for? I worry that that will break Cursorless for people that may have both installed, if you're overriding a parser that we use there

[RedCMD]

I used a custom languageId as well as the .wasm so there wont be any conflicts

I just allowed the option to overwrite for any reason someone might have (including reloading a .wasm without restarting)

[pokey]

Got it. What grammar repo are you using? I wonder if the easiest is just to add that to the parse tree extension. I am a bit worried about breaking Cursorless if extensions start overriding languages, as Cursorless is tightly coupled to the grammars shipped by this repo.

[RedCMD]

my own grammar havn't made a repo for it yet

I personally wont use the overwriting feature so I don't mind it not being included

just want the feature of being able to include a custom .wasm and assigned languageId

[RedCMD]

Im curious did web-tree-sitter work straight out of the box for yous? or did you have to modify some things? (for windows 10)

[pokey]

If I'm understanding your extension correctly, it would override the tree-sitter grammar for a certain subset of json files. This would break Cursorless when a user opens one of these json files.

What would happen if you just used the regular tree-sitter-json parser and then inspect the parse tree to find definitions / references?

[pokey]

Im curious did web-tree-sitter work straight out of the box for yous? or did you have to modify some things? (for windows 10)

It did require a bit of fiddling. Fwiw I originally forked this extension from https://github.com/georgewfraser/vscode-tree-sitter. You could possibly just fork our extension, or cannibalise the code you need for your extension

I'm def not opposed to using this extension for your use case, I'm just hesitant to allow a way to break Cursorless

[RedCMD]

only .tmLanguage.json files (when my extension is enabled)

I assume just reassigning a different languageId id to the file (via the button in the bottom right corner ) would break Cursorless no matter if my extension is enabled or not or if it uses vscode-parse-tree or not

[pokey]

Yeah that's fair. And tbh it might break other extensions that the user has for json support as well 😅. Is there not a way to add support for your features without breaking other json support?

[RedCMD]

not that I know if I don't assigned a new languageId, vscodes builtin json extension will override everything with no way of gaining control of anything

were you able to install the correct node_modules or did you end up having to just copy/paste them from georgewfraser/vscode-tree-sitter?

[pokey]

not that I know if I don't assigned a new languageId, vscodes builtin json extension will override everything with no way of gaining control of anything

Huh. Ok, so I guess for this issue, I'd be fine with supporting parsing files on disk

For #43, I'd be inclined to only allow registering for a language that's not already registered for now

were you able to install the correct node_modules or did you end up having to just copy/paste them from georgewfraser/vscode-tree-sitter?

I think it worked fine 🤷‍♂️. You could try just forking this repo and then migrating the parts you need from https://github.com/RedCMD/TmLanguage-Syntax-Highlighter into the package.json etc.

Or could just use this repo with the changes above; I think either should be ok

[RedCMD]

found vscode.workspace.openTextDocument(path); to work perfectly

I didn't realise that it does not actually open/display the textDocument to the user it only opens it in the background

so it works perfectly at loading the file for this extension to parse

[pokey]

Ok. Not sure I'd recommend that, as it might be a bit heavy to open every document you need for your lsp functionality, but I don't think that affects this extension so that's up to you 😊