usethesource / rascal-language-servers

An LSP server for Rascal which includes an easy-to-use LSP generator for languages implemented in Rascal, and an interactive terminal REPL.
BSD 2-Clause "Simplified" License
10 stars 7 forks source link

Emacs lsp client for rascal-lsp #212

Open credmp opened 1 year ago

credmp commented 1 year ago

Describe the bug

I have been working on integrating rascal support into Emacs. I use emacs-lsp for this task, as it is the defacto implementation of an LSP client for Emacs.

The client sends an initialization request and the server responds with the provided capabilities. From my research into the project one of the capabilities should be completion. When answering the client, the server does not indicate the support for completion.

To Reproduce

Client initialize:

Params: {
  "processId": null,
  "rootPath": "/home/arjen/Projects/rascal-practicum",
  "clientInfo": {
    "name": "emacs",
    "version": "GNU Emacs 28.1.90 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.33, cairo version 1.16.0)\n of 2022-07-14"
  },
  "rootUri": "file:///home/arjen/Projects/rascal-practicum",
  "capabilities": {
    "workspace": {
      "workspaceEdit": {
        "documentChanges": true,
        "resourceOperations": [
          "create",
          "rename",
          "delete"
        ]
      },
      "applyEdit": true,
      "symbol": {
        "symbolKind": {
          "valueSet": [
            1,
            2,
            3,
            4,
            5,
            6,
            7,
            8,
            9,
            10,
            11,
            12,
            13,
            14,
            15,
            16,
            17,
            18,
            19,
            20,
            21,
            22,
            23,
            24,
            25,
            26
          ]
        }
      },
      "executeCommand": {
        "dynamicRegistration": false
      },
      "didChangeWatchedFiles": {
        "dynamicRegistration": true
      },
      "workspaceFolders": true,
      "configuration": true,
      "codeLens": {
        "refreshSupport": true
      },
      "fileOperations": {
        "didCreate": false,
        "willCreate": false,
        "didRename": true,
        "willRename": true,
        "didDelete": false,
        "willDelete": false
      }
    },
    "textDocument": {
      "declaration": {
        "dynamicRegistration": true,
        "linkSupport": true
      },
      "definition": {
        "dynamicRegistration": true,
        "linkSupport": true
      },
      "implementation": {
        "dynamicRegistration": true,
        "linkSupport": true
      },
      "typeDefinition": {
        "dynamicRegistration": true,
        "linkSupport": true
      },
      "synchronization": {
        "willSave": true,
        "didSave": true,
        "willSaveWaitUntil": true
      },
      "documentSymbol": {
        "symbolKind": {
          "valueSet": [
            1,
            2,
            3,
            4,
            5,
            6,
            7,
            8,
            9,
            10,
            11,
            12,
            13,
            14,
            15,
            16,
            17,
            18,
            19,
            20,
            21,
            22,
            23,
            24,
            25,
            26
          ]
        },
        "hierarchicalDocumentSymbolSupport": true
      },
      "formatting": {
        "dynamicRegistration": true
      },
      "rangeFormatting": {
        "dynamicRegistration": true
      },
      "onTypeFormatting": {
        "dynamicRegistration": true
      },
      "rename": {
        "dynamicRegistration": true,
        "prepareSupport": true
      },
      "codeAction": {
        "dynamicRegistration": true,
        "isPreferredSupport": true,
        "codeActionLiteralSupport": {
          "codeActionKind": {
            "valueSet": [
              "",
              "quickfix",
              "refactor",
              "refactor.extract",
              "refactor.inline",
              "refactor.rewrite",
              "source",
              "source.organizeImports"
            ]
          }
        },
        "resolveSupport": {
          "properties": [
            "edit",
            "command"
          ]
        },
        "dataSupport": true
      },
      "completion": {
        "completionItem": {
          "snippetSupport": true,
          "documentationFormat": [
            "markdown",
            "plaintext"
          ],
          "resolveAdditionalTextEditsSupport": true,
          "insertReplaceSupport": true,
          "deprecatedSupport": true,
          "resolveSupport": {
            "properties": [
              "documentation",
              "detail",
              "additionalTextEdits",
              "command"
            ]
          },
          "insertTextModeSupport": {
            "valueSet": [
              1,
              2
            ]
          }
        },
        "contextSupport": true,
        "dynamicRegistration": true
      },
      "signatureHelp": {
        "signatureInformation": {
          "parameterInformation": {
            "labelOffsetSupport": true
          }
        },
        "dynamicRegistration": true
      },
      "documentLink": {
        "dynamicRegistration": true,
        "tooltipSupport": true
      },
      "hover": {
        "contentFormat": [
          "markdown",
          "plaintext"
        ],
        "dynamicRegistration": true
      },
      "selectionRange": {
        "dynamicRegistration": true
      },
      "callHierarchy": {
        "dynamicRegistration": false
      },
      "typeHierarchy": {
        "dynamicRegistration": true
      },
      "publishDiagnostics": {
        "relatedInformation": true,
        "tagSupport": {
          "valueSet": [
            1,
            2
          ]
        },
        "versionSupport": true
      },
      "linkedEditingRange": {
        "dynamicRegistration": true
      }
    },
    "window": {
      "workDoneProgress": true,
      "showDocument": {
        "support": true
      }
    }
  },
  "initializationOptions": null,
  "workDoneToken": "1"
}

The server response:

Result: {
  "capabilities": {
    "textDocumentSync": 1,
    "hoverProvider": true,
    "definitionProvider": true,
    "documentSymbolProvider": true,
    "codeLensProvider": {
      "resolveProvider": null
    },
    "foldingRangeProvider": true,
    "workspace": {
      "workspaceFolders": {
        "supported": true,
        "changeNotifications": true
      }
    },
    "semanticTokensProvider": {
      "legend": {
        "tokenTypes": [
          "entity.name",
          "entity.other.inherited-class",
          "entity.name.section",
          "entity.name.tag",
          "entity.other.attribute-name",
          "variable",
          "variable.language",
          "variable.parameter",
          "variable.function",
          "constant",
          "constant.numeric",
          "constant.language",
          "constant.character.escape",
          "storage.type",
          "storage.modifier",
          "support",
          "keyword",
          "keyword.control",
          "keyword.operator",
          "keyword.declaration",
          "string",
          "comment",
          "invalid",
          "invalid.deprecated",
          "comment.block.documentation",
          "comment.block",
          "comment.single",
          "comment",
          "constant.character.escape",
          "constant.language",
          "constant.numeric.complex.imaginary",
          "constant.numeric.complex.real",
          "constant.numeric.complex",
          "constant.numeric.float.binary",
          "constant.numeric.float.decimal",
          "constant.numeric.float.hexadecimal",
          "constant.numeric.float.octal",
          "constant.numeric.float.other",
          "constant.numeric.float",
          "constant.numeric.integer.binary",
          "constant.numeric.integer.decimal",
          "constant.numeric.integer.hexadecimal",
          "constant.numeric.integer.octal",
          "constant.numeric.integer.other",
          "constant.numeric.integer",
          "constant.numeric",
          "constant.other.placeholder",
          "constant.other",
          "constant",
          "entity.name.class.forward-decl",
          "entity.name.class",
          "entity.name.constant",
          "entity.name.enum",
          "entity.name.function.constructor",
          "entity.name.function.destructor",
          "entity.name.function",
          "entity.name.impl",
          "entity.name.interface",
          "entity.name.label",
          "entity.name.namespace",
          "entity.name.section",
          "entity.name.struct",
          "entity.name.tag",
          "entity.name.trait",
          "entity.name.type",
          "entity.name.union",
          "entity.name",
          "entity.other.attribute-name",
          "entity.other.inherited-class",
          "entity",
          "invalid.deprecated",
          "invalid.illegal",
          "invalid",
          "keyword.control.conditional",
          "keyword.control.import",
          "keyword.control",
          "keyword.declaration.class",
          "keyword.declaration.enum",
          "keyword.declaration.function",
          "keyword.declaration.impl",
          "keyword.declaration.interface",
          "keyword.declaration.struct",
          "keyword.declaration.trait",
          "keyword.declaration.type",
          "keyword.declaration.union",
          "keyword.declaration",
          "keyword.operator.arithmetic",
          "keyword.operator.assignment",
          "keyword.operator.bitwise",
          "keyword.operator.logical",
          "keyword.operator.word",
          "keyword.operator",
          "keyword.other",
          "keyword",
          "markup.bold",
          "markup.deleted",
          "markup.heading",
          "markup.inserted",
          "markup.italic",
          "markup.list.numbered",
          "markup.list.unnumbered",
          "markup.other",
          "markup.quote",
          "markup.raw.block",
          "markup.raw.inline",
          "markup.underline.link",
          "markup.underline",
          "markup",
          "meta.annotation.identifier",
          "meta.annotation.parameters",
          "meta.annotation",
          "meta.block",
          "meta.braces",
          "meta.brackets",
          "meta.class",
          "meta.enum",
          "meta.function-call",
          "meta.function.parameters",
          "meta.function.return-type",
          "meta.function",
          "meta.group",
          "meta.impl",
          "meta.interface",
          "meta.interpolation",
          "meta.namespace",
          "meta.paragraph",
          "meta.parens",
          "meta.path",
          "meta.preprocessor",
          "meta.string",
          "meta.struct",
          "meta.tag",
          "meta.toc-list",
          "meta.trait",
          "meta.type",
          "meta.union",
          "meta",
          "punctuation.accessor",
          "punctuation.definition.annotation",
          "punctuation.definition.comment",
          "punctuation.definition.keyword",
          "punctuation.definition.string.begin",
          "punctuation.definition.string.end",
          "punctuation.definition.variable",
          "punctuation.section.block.begin",
          "punctuation.section.block.end",
          "punctuation.section.braces.begin",
          "punctuation.section.braces.end",
          "punctuation.section.brackets.begin",
          "punctuation.section.brackets.end",
          "punctuation.section.group.begin",
          "punctuation.section.group.end",
          "punctuation.section.interpolation.begin",
          "punctuation.section.interpolation.end",
          "punctuation.section.parens.begin",
          "punctuation.section.parens.end",
          "punctuation.separator.continuation",
          "punctuation.separator",
          "punctuation.terminator",
          "source",
          "storage.modifier",
          "storage.type.class",
          "storage.type.enum",
          "storage.type.function",
          "storage.type.impl",
          "storage.type.interface",
          "storage.type.struct",
          "storage.type.trait",
          "storage.type.union",
          "storage.type",
          "storage.type",
          "string.quoted.double",
          "string.quoted.other",
          "string.quoted.single",
          "string.quoted.triple",
          "string.regexp",
          "string.unquoted",
          "string",
          "support.class",
          "support.constant",
          "support.function",
          "support.module",
          "support.type",
          "text.html",
          "text.xml",
          "text",
          "variable.annotation",
          "variable.function",
          "variable.language",
          "variable.other.constant",
          "variable.other.member",
          "variable.other.readwrite",
          "variable.other",
          "variable.parameter"
        ],
        "tokenModifiers": []
      },
      "full": true
    }
  }
}

Relevant (doom) emacs configuration:

(after! lsp-mode
  (add-to-list 'lsp-language-id-configuration
   '(".*\\.rsc$" . "rascal"))

  (defun lsp-rascal-tcp-connect-to-port ()
    (list
     :connect (lambda (filter sentinel name _environment-fn)
                (let* ((host "localhost")
                       (port 8888)
                       (tcp-proc (lsp--open-network-stream host port (concat name "::tcp"))))

                  (set-process-query-on-exit-flag tcp-proc nil)
                  (set-process-filter tcp-proc filter)
                  (set-process-sentinel tcp-proc sentinel)
                  (cons tcp-proc tcp-proc)))
     :test? (lambda () t)))

  (lsp-register-client
    (make-lsp-client :new-connection (lsp-rascal-tcp-connect-to-port)
                     :major-modes '(rascal-mode)
                     :server-id 'rascal-lsp)))

(add-to-list 'auto-mode-alist '("\\.rsc$" . rascal-mode))

Expected behavior

The server should indicate support for completion.

jurgenvinju commented 1 year ago

First of all: yes! Thanks for putting time in this. If you need help with the repl extensions we made and the ideservices, just shout out.

Will look into this current issue tomorrow.

jurgenvinju commented 1 year ago

By the way does the emacs LSP client support Semantic highlighting? Because we depend on that to mimick highlighters based on Rascal generated parsers.

credmp commented 1 year ago

The client supports the semantic highlighting indeed. Relevant changelog: https://emacs-lsp.github.io/lsp-mode/blog/2020/7.0-release/#semantic-highlights

credmp commented 1 year ago

When I enable semantic-highlighting the following are the client capabilities:

ClientCapabilities [
  workspace = WorkspaceClientCapabilities [
    applyEdit = true
    workspaceEdit = WorkspaceEditCapabilities [
      documentChanges = true
      resourceOperations = ArrayList (
        "create",
        "rename",
        "delete"
      )
      failureHandling = null
      normalizesLineEndings = null
      changeAnnotationSupport = null
    ]
    didChangeConfiguration = null
    didChangeWatchedFiles = DidChangeWatchedFilesCapabilities [
      relativePatternSupport = null
      dynamicRegistration = true
    ]
    symbol = SymbolCapabilities [
      symbolKind = SymbolKindCapabilities [
        valueSet = ArrayList (
          File,
          Module,
          Namespace,
          Package,
          Class,
          Method,
          Property,
          Field,
          Constructor,
          Enum,
          Interface,
          Function,
          Variable,
          Constant,
          String,
          Number,
          Boolean,
          Array,
          Object,
          Key,
          Null,
          EnumMember,
          Struct,
          Event,
          Operator,
          TypeParameter
        )
      ]
      tagSupport = null
      resolveSupport = null
      dynamicRegistration = null
    ]
    executeCommand = ExecuteCommandCapabilities [
      dynamicRegistration = false
    ]
    workspaceFolders = true
    configuration = true
    semanticTokens = SemanticTokensWorkspaceCapabilities [
      refreshSupport = false
    ]
    codeLens = CodeLensWorkspaceCapabilities [
      refreshSupport = true
    ]
    fileOperations = FileOperationsWorkspaceCapabilities [
      didCreate = false
      willCreate = false
      didRename = true
      willRename = true
      didDelete = false
      willDelete = false
      dynamicRegistration = null
    ]
    inlayHint = null
    inlineValue = null
    diagnostics = null
  ]
  textDocument = TextDocumentClientCapabilities [
    synchronization = SynchronizationCapabilities [
      willSave = true
      willSaveWaitUntil = true
      didSave = true
      dynamicRegistration = null
    ]
    completion = CompletionCapabilities [
      completionItem = CompletionItemCapabilities [
        snippetSupport = true
        commitCharactersSupport = null
        documentationFormat = ArrayList (
          "markdown",
          "plaintext"
        )
        deprecatedSupport = true
        preselectSupport = null
        tagSupport = null
        insertReplaceSupport = true
        resolveSupport = CompletionItemResolveSupportCapabilities [
          properties = ArrayList (
            "documentation",
            "detail",
            "additionalTextEdits",
            "command"
          )
        ]
        insertTextModeSupport = CompletionItemInsertTextModeSupportCapabilities [
          valueSet = ArrayList (
            AsIs,
            AdjustIndentation
          )
        ]
        labelDetailsSupport = null
      ]
      completionItemKind = null
      contextSupport = true
      insertTextMode = null
      completionList = null
      dynamicRegistration = true
    ]
    hover = HoverCapabilities [
      contentFormat = ArrayList (
        "markdown",
        "plaintext"
      )
      dynamicRegistration = true
    ]
    signatureHelp = SignatureHelpCapabilities [
      signatureInformation = SignatureInformationCapabilities [
        documentationFormat = null
        parameterInformation = ParameterInformationCapabilities [
          labelOffsetSupport = true
        ]
        activeParameterSupport = null
      ]
      contextSupport = null
      dynamicRegistration = true
    ]
    references = null
    documentHighlight = null
    documentSymbol = DocumentSymbolCapabilities [
      symbolKind = SymbolKindCapabilities [
        valueSet = ArrayList (
          File,
          Module,
          Namespace,
          Package,
          Class,
          Method,
          Property,
          Field,
          Constructor,
          Enum,
          Interface,
          Function,
          Variable,
          Constant,
          String,
          Number,
          Boolean,
          Array,
          Object,
          Key,
          Null,
          EnumMember,
          Struct,
          Event,
          Operator,
          TypeParameter
        )
      ]
      hierarchicalDocumentSymbolSupport = true
      tagSupport = null
      labelSupport = null
      dynamicRegistration = null
    ]
    formatting = FormattingCapabilities [
      dynamicRegistration = true
    ]
    rangeFormatting = RangeFormattingCapabilities [
      dynamicRegistration = true
    ]
    onTypeFormatting = OnTypeFormattingCapabilities [
      dynamicRegistration = true
    ]
    declaration = DeclarationCapabilities [
      linkSupport = true
      dynamicRegistration = true
    ]
    definition = DefinitionCapabilities [
      linkSupport = true
      dynamicRegistration = true
    ]
    typeDefinition = TypeDefinitionCapabilities [
      linkSupport = true
      dynamicRegistration = true
    ]
    implementation = ImplementationCapabilities [
      linkSupport = true
      dynamicRegistration = true
    ]
    codeAction = CodeActionCapabilities [
      codeActionLiteralSupport = CodeActionLiteralSupportCapabilities [
        codeActionKind = CodeActionKindCapabilities [
          valueSet = ArrayList (
            "",
            "quickfix",
            "refactor",
            "refactor.extract",
            "refactor.inline",
            "refactor.rewrite",
            "source",
            "source.organizeImports"
          )
        ]
      ]
      isPreferredSupport = true
      disabledSupport = null
      dataSupport = true
      resolveSupport = CodeActionResolveSupportCapabilities [
        properties = ArrayList (
          "edit",
          "command"
        )
      ]
      honorsChangeAnnotations = null
      dynamicRegistration = true
    ]
    codeLens = null
    documentLink = DocumentLinkCapabilities [
      tooltipSupport = true
      dynamicRegistration = true
    ]
    colorProvider = null
    rename = RenameCapabilities [
      prepareSupport = true
      prepareSupportDefaultBehavior = null
      honorsChangeAnnotations = null
      dynamicRegistration = true
    ]
    publishDiagnostics = PublishDiagnosticsCapabilities [
      relatedInformation = true
      tagSupport = Either [
        left = null
        right = DiagnosticsTagSupport [
        valueSet = ArrayList (
          Unnecessary,
          Deprecated
        )
      ]
      ]
      versionSupport = true
      codeDescriptionSupport = null
      dataSupport = null
    ]
    foldingRange = null
    typeHierarchy = TypeHierarchyCapabilities [
      dynamicRegistration = true
    ]
    callHierarchy = CallHierarchyCapabilities [
      dynamicRegistration = false
    ]
    selectionRange = SelectionRangeCapabilities [
      dynamicRegistration = true
    ]
    semanticTokens = SemanticTokensCapabilities [
      requests = SemanticTokensClientCapabilitiesRequests [
        range = Either [
          left = true
          right = null
        ]
        full = Either [
          left = true
          right = null
        ]
      ]
      tokenTypes = ArrayList (
        "comment",
        "keyword",
        "string",
        "number",
        "regexp",
        "operator",
        "namespace",
        "type",
        "struct",
        "class",
        "interface",
        "enum",
        "typeParameter",
        "function",
        "method",
        "member",
        "property",
        "event",
        "macro",
        "variable",
        "parameter",
        "label",
        "enumConstant",
        "enumMember",
        "dependent",
        "concept"
      )
      tokenModifiers = ArrayList (
        "declaration",
        "definition",
        "implementation",
        "readonly",
        "static",
        "deprecated",
        "abstract",
        "async",
        "modification",
        "documentation",
        "defaultLibrary"
      )
      formats = ArrayList (
        "relative"
      )
      overlappingTokenSupport = null
      multilineTokenSupport = null
      serverCancelSupport = null
      augmentsSyntaxTokens = null
      dynamicRegistration = true
    ]
    moniker = null
    linkedEditingRange = LinkedEditingRangeCapabilities [
      dynamicRegistration = true
    ]
    inlayHint = null
    inlineValue = null
    diagnostic = null
  ]
  notebookDocument = null
  window = WindowClientCapabilities [
    workDoneProgress = true
    showMessage = null
    showDocument = ShowDocumentCapabilities [
      support = true
    ]
  ]
  general = null
  experimental = null
]
jurgenvinju commented 1 year ago

Ok. We didn't implement the completion capability yet. It works in the terminal in Rascal because that is implemented independently of the IDE. The completion support in the VScode editor is a language agnostic default feature of VScode.

credmp commented 1 year ago

Ah, I did a quick search of the code for the word 'completion' so I made the assumption that it was working. So for now it is just the semantic highlighting and then the IDEServices/REPL. Those are things I need to figure out how to do still, but then I suggest we close this issue and revisit the functionality when it is implemented in the LS. Agreed?

jurgenvinju commented 1 year ago

Let's turn this into a feature request. We can leave it open until we get to it. Of course we'll do it for rascal and DSLs in Rascal at once.

credmp commented 1 year ago

Alright, that is beautiful! I will finish up the Emacs integration (making a new major mode with LSP) and then will look at the integrations of the repl.

DavyLandman commented 1 year ago

This sounds like a nice project.

If you want to know which capabilities the rascal-lsp supports, take a look here:

https://github.com/usethesource/rascal-language-servers/blob/13e9185f01e15bf812b027403dec0655f06c2d07/rascal-lsp/src/main/java/org/rascalmpl/vscode/lsp/rascal/RascalTextDocumentService.java#L143-L149

Note, we also extend the LSP protocol a bit, see:

Have fun!

DavyLandman commented 1 year ago

Let's turn this into a feature request. We can leave it open until we get to it. Of course we'll do it for rascal and DSLs in Rascal at once.

For rascal we already have an issue tracking this: https://github.com/usethesource/rascal-language-servers/issues/12

For parametric the story is even stranger. Our LanguageService adt contains an entry for the completer: https://github.com/usethesource/rascal-language-servers/blob/13e9185f01e15bf812b027403dec0655f06c2d07/rascal-lsp/src/main/rascal/util/LanguageServer.rsc#L68

But we ignore that in the implementation. So that's not good. (I've made an issue tracking it #213, as I think both cases are quite different in their implementation and trade-offs.)

credmp commented 1 year ago

Thanks @DavyLandman! I will post back with my progress on the mode for Emacs. Being able to properly edit Rascal and have proper highlighting is already a very nice thing. I believe the integration in emacs itself for the repl (send to repl / click on urls) should be interesting. In the Clojure world there is a similar effort with nrepl, perhaps it is possible to leverage that functionality.

jurgenvinju commented 1 year ago

@credmp technically starting up a rascal terminal is pretty easy; it's a jvm system call and you have to pass in the right ports such that the terminal can connect with the LSP server. After that all the IDE integrations run via the existing LSP connection. For example (and this one is rather complex to show all the facets involved):

  1. registerLanguage is typed in a terminal REPL by a user
  2. registerLanguage JSON-RPC message goes from terminal to Rascal LSP server
  3. registerLanguage JSON-RPC message goes from Rascal LSP server to LSP client via the basic LSP connection
  4. client code registers new extension with the Parameterized LSP server (second LSP server special for DLSs)
  5. registerLanguage JSON-RPC message is sent from client to the Parameterized LSP server
  6. The parametrized LSP loads all the code to start serving an LSP for the new language

There are also simpler messages outside of the standard protocol that are handled by the client, such as registering new URI schemes. The extended JSON-RPC messages and notifications labels are alll prefixed with "rascal/".