How-to: Registering a language in VScode?

aukeroorda commented 2 years ago

Context Hi - I'm trying to move a Rascal-eclipse project to Rascal-lsp. The Rascal-eclipse project uses the registerLanguage() function from util::IDE to register the language, after which the Eclipse environment uses the generated language plugin on files with the specified file extension. However, in the new VScode environment, there are some changes required to make this work. The Rascal update release notes state this as well:

Porting Rascal’s IDE generator features to VScode has had minor impact on the modules in the standard library concerning interaction with DSLs and interactions with the IDE (util::IDEServices, util::Monitor and util::IDE). The old API still works and it is backward compatible in Eclipse. However, if you want to port your DSL to VScode, there are minor changes in how to wrap your extension and new interaction possibilities with the IDE which are not present in Eclipse.

However, it is unclear to me which changes are required, and I have not managed to register my DSL again.

Current attempt I dug around in the Rascal repositories, and found a file implementing the language server functionalities and registering the language for the demo language Pico. To start off, I copied this file to my-app/src/CorelLanguageServer.rsc and changed the code to use the Syntax of the example project, instead of the Pico language.

The editor shows no problems with this file. It can be imported without problems in the Rascal Terminal:

However, when you add an additional import statement import util::NonExisting;, we can suddenly see that the file is full of undefined modules and references to names that cannot be resolved:

Problems found

```json [{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Reference to name `util::LanguageServer` cannot be resolved", "startLineNumber": 6, "startColumn": 8, "endLineNumber": 6, "endColumn": 28 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined module `util::LanguageServer`", "startLineNumber": 6, "startColumn": 8, "endLineNumber": 6, "endColumn": 28 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined module `util::NonExisting`", "startLineNumber": 7, "startColumn": 8, "endLineNumber": 7, "endColumn": 25 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Reference to name `util::NonExisting` cannot be resolved", "startLineNumber": 7, "startColumn": 8, "endLineNumber": 7, "endColumn": 25 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined module `util::Reflective`", "startLineNumber": 8, "startColumn": 8, "endLineNumber": 8, "endColumn": 24 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Reference to name `util::Reflective` cannot be resolved", "startLineNumber": 8, "startColumn": 8, "endLineNumber": 8, "endColumn": 24 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined `LanguageService`", "startLineNumber": 12, "startColumn": 5, "endLineNumber": 12, "endColumn": 20 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Types type[&T] and Tree(str,loc) do not match", "startLineNumber": 13, "startColumn": 12, "endLineNumber": 15, "endColumn": 6 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined `outliner`", "startLineNumber": 16, "startColumn": 5, "endLineNumber": 16, "endColumn": 13 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined `summarizer`", "startLineNumber": 17, "startColumn": 5, "endLineNumber": 17, "endColumn": 15 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined `lenses`", "startLineNumber": 18, "startColumn": 5, "endLineNumber": 18, "endColumn": 11 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined `executor`", "startLineNumber": 19, "startColumn": 5, "endLineNumber": 19, "endColumn": 13 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined `inlayHinter`", "startLineNumber": 20, "startColumn": 5, "endLineNumber": 20, "endColumn": 16 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined `DocumentSymbol`", "startLineNumber": 23, "startColumn": 6, "endLineNumber": 23, "endColumn": 20 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined `symbol`", "startLineNumber": 24, "startColumn": 8, "endLineNumber": 24, "endColumn": 14 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Expected 1 argument(s), found 0", "startLineNumber": 24, "startColumn": 30, "endLineNumber": 24, "endColumn": 35 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined `symbol`", "startLineNumber": 25, "startColumn": 11, "endLineNumber": 25, "endColumn": 17 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined `\\variable`", "startLineNumber": 25, "startColumn": 34, "endLineNumber": 25, "endColumn": 43 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined `Summary`", "startLineNumber": 28, "startColumn": 1, "endLineNumber": 28, "endColumn": 8 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined `summary`", "startLineNumber": 33, "startColumn": 12, "endLineNumber": 33, "endColumn": 19 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined keyword argument `title`; no other keyword parameters available", "startLineNumber": 43, "startColumn": 86, "endLineNumber": 43, "endColumn": 118 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined `InlayHint`", "startLineNumber": 45, "startColumn": 6, "endLineNumber": 45, "endColumn": 15 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined `hint`", "startLineNumber": 51, "startColumn": 13, "endLineNumber": 51, "endColumn": 17 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined `\\type`", "startLineNumber": 51, "startColumn": 48, "endLineNumber": 51, "endColumn": 53 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined `registerLanguage`", "startLineNumber": 62, "startColumn": 5, "endLineNumber": 62, "endColumn": 21 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined `language`", "startLineNumber": 63, "startColumn": 9, "endLineNumber": 63, "endColumn": 17 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 8, "message": "Undefined `pathConfig`", "startLineNumber": 64, "startColumn": 13, "endLineNumber": 64, "endColumn": 23 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 4, "message": "Unused import of `util::LanguageServer`", "startLineNumber": 6, "startColumn": 1, "endLineNumber": 6, "endColumn": 29 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 4, "message": "Unused import of `util::NonExisting`", "startLineNumber": 7, "startColumn": 1, "endLineNumber": 7, "endColumn": 26 },{ "resource": "/Users/auke/rascal-lsp-skeletons/my-app/src/CorelLanguageServer.rsc", "owner": "_generated_diagnostic_collection_name_#4", "severity": 4, "message": "Unused import of `util::Reflective`", "startLineNumber": 8, "startColumn": 1, "endLineNumber": 8, "endColumn": 25 }] ```

To try to solve this, I mimicked the structure around the demo Pico LanguageServer.rsc file, and added the file LanguageServer.rsc in a directory util. This however does not seem to solve the issue. I've also tried to solve this by adding rascal-core as a dependency to the project, but this does also not seem to solve the issue. An overview of the changes made can be see in this PR, and the current state of my attempt can be cloned as follows:

git clone -b language-server https://github.com/aukeroorda/rascal-lsp-skeletons.git

Expected behavior I expected to be able to import the modules util::LanguageServer and util::Reflective just like the LanguageServer.rsc in the Pico demo does, after which I can register my language.

Note:

At first I didn't know about the problems with the broken imports as shown above, and only ran into a problem when opening a .pico file, in which case the following error was shown:

Start of previous issue details

Parsing failed: Failure to import required module LanguageServer

"Old" issue can be reproduced by

Steps to reproduce the behavior:

Clone git clone -b language-server https://github.com/aukeroorda/rascal-lsp-skeletons.git
Open the file my-app/src/CorelLanguageServer.rsc
Click "Run in new Rascal terminal" above the function void main()
Verify that the Rascal terminal says "ok" for both calls:
```
rascal>import CorelLanguageServer;
ok
rascal>main()
ok
```
Open the file my-app/src/demo.pico
See error in the problems section of VScode

End of previous issue details

Additional information

Output of `mvn dependency:build-classpath`

``` /Users/auke/.m2/repository/junit/junit/4.11/junit-4.11.jar: /Users/auke/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar: /Users/auke/.m2/repository/io/usethesource/vallang/0.14.4/vallang-0.14.4.jar: /Users/auke/.m2/repository/io/usethesource/capsule/0.7.0/capsule-0.7.0.jar: /Users/auke/.m2/repository/org/apache/commons/commons-compress/1.21/commons-compress-1.21.jar: /Users/auke/.m2/repository/org/tukaani/xz/1.9/xz-1.9.jar: /Users/auke/.m2/repository/com/github/luben/zstd-jni/1.5.1-1/zstd-jni-1.5.1-1.jar: /Users/auke/.m2/repository/com/github/ben-manes/caffeine/caffeine/3.0.4/caffeine-3.0.4.jar: /Users/auke/.m2/repository/com/google/errorprone/error_prone_annotations/2.9.0/error_prone_annotations-2.9.0.jar: /Users/auke/.m2/repository/org/checkerframework/checker-qual/3.21.0/checker-qual-3.21.0.jar: /Users/auke/.m2/repository/org/rascalmpl/rascal/0.23.0/rascal-0.23.0.jar: /Users/auke/.m2/repository/org/rascalmpl/rascal-core/0.4.21/rascal-core-0.4.21.jar: /Users/auke/.m2/repository/frinklang/frink/22.4.18/frink-22.4.18.jar ```

Output of `getProjectPathConfig(|project://my-app|)`

Desktop (please complete the following information):

Context: Rascal-lsp plugin as installed from the marketplace.
Rascal Version: 0.23.1
Rascal-lsp Version: 2.1.1-SNAPSHOT

DavyLandman commented 2 years ago

Thank you of the detail report.

The first one is already tracked: https://github.com/usethesource/rascal/issues/1570 as soon as you add an non-existing module, rascal-core typechecker sometimes stops a bit too early.

You don't (and shouldn't) take a dependency on rascal-core, but maybe you should try to take a maven dependency on rascal-lsp. For example:

  <dependency>
      <groupId>org.rascalmpl</groupId>
      <artifactId>rascal-lsp</artifactId>
      <version>2.1.0</version>
    </dependency>

And drop the invalid import.

Finally, this API is used incorrectly:

void main() {
    registerLanguage(
        language(
            pathConfig(),
            "Pico",
            "pico",
            "LanguageServer",
            "picoLanguagecontributor"
        )
    );
}

it should be:

void main() {
    registerLanguage(
        language(
            pathConfig(),
            "StateMachineLanguage",
            "pico",
            "CorelLanguageServer", // module to import for the DSL
            "picoLanguagecontributor" // function to call with the contributions of this DSL
        )
    );
}

aukeroorda commented 2 years ago

Thanks for your reply!

The first one is already tracked: https://github.com/usethesource/rascal/issues/1570 as soon as you add an non-existing module, rascal-core typechecker sometimes stops a bit too early.

This is good to know! Somehow I thought it was the other way around; that when you included a non-existing module, the checker would "wake up" and suddenly also find out that other modules could not be resolved.

You don't (and shouldn't) take a dependency on rascal-core, but maybe you should try to take a maven dependency on rascal-lsp. For example:

I've removed this and replaced it by a rascal-lsp dependency in the pom.xml file, and updated the RASCAL.MF file.

Finally, I have also updated CorelLanguageServer.rsc so that the main function uses the API correctly:

void main() {
    registerLanguage(
        language(
            pathConfig(),
            "StateMachineLanguage",     // Name of the language
            "pico",                     // Extension of source files
            "CorelLanguageServer",      // Module that is imported for the DSL
            "picoLanguagecontributor"   // Main function to call with the contributions of this DSL
        )
    );
}

This prompted another look at the values of the parameters, and the first value, pathConfig() turned out to be the problem. This empty constructor had to be replaced by getProjectPathConfig(|project://my-app/|), and then it works!

So the resulting registerLanguage call looks as follows:

void main() {
    registerLanguage(
        language(
            getProjectPathConfig(|project://my-app/|),
            "StateMachineLanguage",     // Name of the language
            "pico",                     // Extension of source files
            "CorelLanguageServer",      // Module that is imported for the DSL
            "picoLanguagecontributor"   // Main function to call with the contributions of this DSL
        )
    );
}

For reference: This PR contains all the changes made to get the parametric language server working for the DSL. It turns out that the dependency on rascal-lsp was not needed either, so that is removed here.

DavyLandman commented 2 years ago

looks good, I forgot to check the path config part, that indeed is also crucial so that the evaluator for DSLs can find your modules.

usethesource / rascal-language-servers

How-to: Registering a language in VScode? #157

Start of previous issue details

End of previous issue details