Notebook support for upcoming LSP 3.17.0 spec

[jackos]

Please see draft PR for adding Notebook support to lsp-types for the upcoming changes to the LSP spec: https://github.com/gluon-lang/lsp-types/pull/229

To get this working there will need to be some extra, but very simple code generation, see here: https://github.com/codebooknet/codebook/blob/main/src/languages/rust.ts

Would the rust-analyzer project be interested in merging this functionality if it doesn't affect the existing codebase, or should it be a separate fork? If there is no interest I can create a separate language server that only registers NotebookDocumentSyncCapabilities, but it would be preferable to have it inside rust-analyzer, any notebook extension in VS Code will magically get full LSP Rust support, which will make writing documentation and learning/experimenting with Rust a great user experience.

[ChayimFriedman2]

rust-analyzer requires a Cargo workspace to provide rich experience. Notebooks are usually standalone files.

That said, there might be a way to create a Cargo workspace on-the-fly or something, I'm not sure.

[bjorn3]

The only rust notebook I am aware of is evcxr which internally uses rust-analyzer for determining the types of variables to persist across notebook cells. It uses rust-analyzer as library because lsp doesn't provide the interfaces it needs. Maybe it would make sense to add support for being an lsp server to evcxr. It needs integration with evcxr anyway as evcxr changes the code you enter. Rust doesn't allow expressions at the top level so bare rust-analyzer doesn't either. Evcxr could present rust-analyzer with the actual code it is going to run and then forward the response from rust-analyzer.

[jackos]

@bjorn3 @ChayimFriedman2 I've developed a notebook using VS Code native notebooks that supports Rust here: https://marketplace.visualstudio.com/items?itemName=codebook.codebook

It's very simple, it just does some minor code generation to create a fn main if it doesn't exist, moves the "use" statements to generate a cargo.toml for external code and uses the local Rust toolchain to execute code.

I've got rust-analyzer working in one cell with all the features, but to use multiple cells there's some code generation required, I've come up with a potential solution writing a custom client and using an intermediary file, and synching that with rust-analyzer, but it's hacky and the best way would be to actually use this LSP notebook spec that Microsoft has developed, adding it to rust-analyzer

It's a lot of work as rust-analyzer is a large project, so I wanted to check if a functional pull request for this would be accepted.

[bjorn3]

The problem is that every notebook runner for rust does different wrapping of the notebook cells to turn it into valid rust. This means that rust-analyzer can't directly implement the notebook lsp api, but needs integration with each notebook runner. Having the notebook runner implement the notebook lsp api and forward the wrapped code to rust-analyzer (either through lsp or as rust library) doesn't require any integration with specific notebook runners.

[jackos]

This means that rust-analyzer can't directly implement the notebook lsp API

My thoughts were that it's so simple to turn Notebooks into valid rust code that all notebook runners would be implementing their code generation in the same way, which would match what I implement in rust-analyzer

But yeah you're right, the best approach is to probably to generate the code client-side and sync that, I just need to figure out how to:

• Get VS Code to generate an intermediary file that's stored in memory not disk, and updates with each keystroke, for all the fast responses
• Get rust-analyzer to sync that intermediary file inside a generated workspace with a Cargo.toml
• Apply the changes that are being sent back to each notebook cell with an offset for the positions

Shouldn't be too hard, and I think more simple than all the additions that would be required in rust-analyzer.

Thanks for your thoughts, I'll close this issue.