search

jupyter-lsp / jupyterlab-lsp

Coding assistance for JupyterLab (code navigation + hover suggestions + linters + autocompletion + rename) using Language Server Protocol
https://jupyterlab-lsp.readthedocs.io
BSD 3-Clause "New" or "Revised" License
1.81k stars 148 forks source link

Use in-memory file system for virtual documents (if available) #356

Open krassowski opened 4 years ago

krassowski commented 4 years ago

What are you trying to do?

Improve the performance by storing the virtual documents on ramdisk. I've used ramdisk before from Python, but this time looking at pyfilesystem's memoryfs hoping it will give us cheap cross-platform support:

How is it done today, and what are the limits of current practice?

We are writing and reading a lot form the disk. For those who have SSD disks the speed is ok, but it must be a pain for others; even for those who have SSD, it would help them as the longevity of the SSDs will increase.

This might require that the virtual file system is symlinked to the project (or maybe not - we will see).

Related to slow autocompletion (#272) and other improvements to virtual documents #353 , #336

bollwyvl commented 4 years ago

As the complexity of this goes up, something that starts bringing in lots of dependencies might need to become managed as its own package (e.g. jupyter-lsp-vdocs) which depends on jupyter-lsp-core... and/or we might need to collaborate more closely with the debugger folks. I can't imagine two things try to do this at the same time on one computer is any fun.

the proxy kernel approach certainly complicates things, as to when this work should be done, but might improve this situation, as at least one wouldn't have to bounce the server to try out a new feature...

krassowski commented 4 years ago

Right, it would be useful to disable this workaround if one only uses LSP servers/features that do not require a file on disk.

Had a look into pyfilesystem and unfortunately is not fit for the task as it does not give the on-disk path for the memory filesystem (i.e. it seems it uses a custom implementation rather than standard ramdisk which can be mounted to any directory) so we cannot give a valid path to the LSP servers.

It seems that using Linux ramdisk and ImDisk/SoftPerfect RAM Disk for Windows would be the way forward. This all should be wrapped in a huge try-catch and fallback to a static directory or tmpdir.

bollwyvl commented 4 years ago

Maybe it's worth looking at what pygls does?

https://github.com/openlawlibrary/pygls/blob/master/pygls/workspace.py

Again, I wouldn't be sad if we took that as a dependency, if it provides value... while it also doesn't write to disk, it might be a better scaffolding than doing it ourselves/differently...