Connecting to LSP with docker-in-docker setups.

[paul]

Description

We've got a Rails app with some expensive and complicated dependencies (libarrow, among other things), so we run the app in docker-compose. Additionally, for most of our devs using macs, we've started using gitpod to develop the app. They run their own gidpod container on their own infrastructure, and expose the docker socket to that container, and we can docker-compose our app from there. Then vscode can connect to the gitpod docker infrastructure and interact with the files, etc...

I'm trying to figure out how to convince the vscode that connects to the gitpod container to run the ruby-lsp command within our dev container. I can write a wrapper script that would do something like docker compose run -it dev ruby_lsp, but I'm not sure how to get the extension to run my wrapper. Also, its gonna try to boot rails, so I'd need to wrap that command, too.

Not a bug report exactly, but maybe a feature request, to allow for a setting in the vscode extension that would let me override the command used to invoke ruby_lsp. (and another to control how it boots the rails app). Alternatively, there's other issues on here to expose a socket or port for vscode to connect to, so that might work if we can start ruby-lsp independently as part of docker-compose, and configure vscode to connect there.

Reproduction steps

1. Have an app with a complicated set of dependencies, such that it can't bundle install or boot on a local machine or in a "normal" container, only within a purpose-built container with the deps needed.
2. Get vscode to run ruby_lsp inside that container.

[willtcarey]

We've got a similar setup to what you're describing, except we remote into linux machines from our macOS machines (using the VS Code Remote SSH extensino) and then run our app code in docker.

Ruby LSP appears to have support for everything running inside of a devcontainer, but not this case where you are running VS Code outside of the container which is running your app code/LSP instance.

I managed to make the VS Code extension run my custom wrapper script by running the Ruby LSP: Select ruby version manager -> Change manual ruby configuration then picking my wrapper script. However, it's a bit odd that the extension only stores that in a workspaceState rather than a configuration option. I'd like to share that configuration with my team, but I don't know a way to do that with workspaceState.

https://github.com/Shopify/ruby-lsp/blob/caa4a4038f951f85c05a5a8bb363804065adddc1/vscode/src/ruby.ts#L106-L136

The second thing that's odd here is that I could avoid going the workspaceState route at all if I could set my version manager to none and set a custom rubyExecutablePath. But the custom ruby path is not passed to the none version manager on the first pass.

https://github.com/Shopify/ruby-lsp/blob/caa4a4038f951f85c05a5a8bb363804065adddc1/vscode/src/ruby.ts#L293-L297

So that means that the extension attempts to execute ruby on my machine outside of docker. I have a system installed ruby so the command works and the extension tries to go down that activation path before erroring out because that's not the correct ruby it should be using. I think that we should always pass the custom rubyExecutablePath if it is set for consistency.

Ok, so while I think some of the above things should be fixed, I've managed to work around them by using a custom version manager which ran a script to add some custom ruby, gem and bundler shims to my path which executed the respective command inside my docker container. This seems like a temporary hack, but at least it got me to this point. When I did that, however, I got errors because my Dockerfile adds LD_PRELOAD environment variables for jemalloc, and the extension was trying to add that to my shell environment when executing other commands, but that jemalloc library doesn't exist on the host machine, so it immediately errors. Maybe we could configure what env params get sanitized during env sanitization process?

So it appears there are several layers to this and several different locations where I need to tell ruby-lsp that I want to run it inside my container. I believe this should technically work as the LSP protocol runs through plain text communication on STDOUT (please correct me if I'm wrong there). But I'm definitely running into several walls. I'm not sure the best way to approach this as it looks like it would take changes in several places, but I wanted to leave my experience of where I got to.

[butsjoh]

I have a different use case but i think i boils down to the same issue. We have a monorepo with multiple projects (some of them are written in ruby). We run the projects through docker-compose. We have vscode locally installed with the ruby-lsp extension and currently we can only get things to work if we locally install ruby. It would be nice if we could use the ruby that is already available in the docker containers.

[vinistock]

For simpler docker setups, VS Code already supports connecting the editor directly to the container, which makes everything in the editor works, including the integrated terminal. We have docs for this use case https://github.com/Shopify/ruby-lsp/tree/main/vscode#developing-on-containers.

For more complex setups, like the one mentioned, I believe it ultimately boils down to the LSP supporting connecting through sockets #1782. The LSP has to run inside wherever Ruby and your dependencies are installed and where the code lives - since it needs to read it all from disk to index the declarations.

If someone is interested in taking a stab at this, my understanding is that the server needs to support CLI arguments to connect via ports. I'm not exactly sure if the language client package still owns the life cycle of the process in this scenario or if there's some custom code that needs to exist to make it work. It might be worth checking how language server implementations for other languages support this.