Multiple nrepl connections

kstehn commented 5 years ago

Hey,

first of all, awesome extension.

I want to ask if there are any plans to make it possible to connect with multiple nrepls. I looked in the code. I think if you add the Project name to the terminal name it should be possible. But I have no idea how to do it and what implication it would have, especially with autocomplete and Jump to Defenition etc.

PEZ commented 5 years ago

I think this should be possible. But like you I haven't thought the implications through yet.

bpringe commented 4 years ago

I want to take a stab at some requirements / expectations for this. Some of my understanding may be a bit off, as I've only recently got into ClojureScript dev (specifically using shadow-cljs), and had only really worked with Clojure before.

Calva should support multiple REPL connections
I think at least managing 1 clj and 1 cljs REPL per project makes sense (and maybe only 1 each)
Each connection would be stored with the project name and -clj or -cljs appended to identify it
Scenario 1 - A single project that includes clj and cljs, using shadow-cljs
1. I open my-proj in vs code which contains a deps.edn file and a shadow-cljs.edn file
2. I run jack-in and select Clojure CLI, which starts a clojure repl in a terminal and connects Calva to it
3. I run jack-in again and select shadow-cljs, which starts a clojurescript repl in another terminal, and connects Calva to it
4. Both repl connections are stored in Calva under names such as my-proj-clj and my-proj-cljs
5. I open src/my_proj/core.clj and run a command to evaluate a form. Calva sees the extension is clj and the file's namespace starts with my-proj, so it sends the form to the connection called my-proj-clj.
6. I then open src/my_proj/client/core.cljs. Calva sees the extension is cljs and the file's namespace starts with my-proj so it sends the form to the connection called my-proj-cljs
  - Note: There's no manual switching of connections by the user, Calva knows where to send the forms. Of course there may be exceptions such as namespaces that don't match a project name, but these can be handled fairly easily I think.
Scenario 2 - A monorepo which contains multiple projects
1. I open the repo root in vs code
2. I navigate to some file inside one of the projects in the repo
3. I run jack-in, which starts a clj or cljs repl accordingly (if one doesn't exist for this project), and it searches from the current directory up for the next closest project file, which it uses for starting the repl.
  - Note: Everything else is the same as in scenario 1 above.

I know this all could require possibly significant refactoring of some code around how we handle nrepl connections, but it's definitely doable.

Thoughts welcome, @kstehn @PEZ!

kstehn commented 4 years ago

I think most of it is fine. But i don't think it will be enough to say that my_proj is the only namespace in a project so this must be the repl.

For example in our project we have something like: src/clj/my_project/core.clj src/clj/commons/util.clj and same then goes for the cljs part

And one question or thing to be considered: How do we handle a mono repo with multiple full-stack projects? Or more like can we give the user to open a cljs repl after starting a clj repl.

What i have in mind i start my full-stack project 1. With server-part and client and i use for example lein with figwheel. And i also have a project 2 similar to the first one and start this. What i read from Scenario 2 in both cases i can start clj OR cljs repl.

So do i need to Jack-In again to get a cljs repl? Or could we just give a command to open a cljs repl connection afterwards if possible?

bpringe commented 4 years ago

But i don't think it will be enough to say that my_proj is the only namespace in a project so this must be the repl.

This is what I meant by exceptions we'd have to deal with. You bring up a good point with your example. What we could do instead of label repl connections by project name is to label them by project root path.

So for example if all you have open is that one project. Assuming the root is /home/kstehn/my-project then when you jack-in the repl connection is labeled with that path. When you evaluate a form in src/clj/commons/util.clj then Calva sees .clj and the path of the current document and sends it to the correct repl.

How do we handle a mono repo with multiple full-stack projects? Or more like can we give the user to open a cljs repl after starting a clj repl.

I think this would work the same as it does now basically, in terms of tools like figwheel, where the cljs repl can be started from the clj repl. Custom jack-in sequences can still be used, etc.

What i read from Scenario 2 in both cases i can start clj OR cljs repl.

I was speaking more to my scenario of using shadow-cljs, which needs to be (or is recommended to be) started separately using the npm module. But I think the current project types which combine server+client would still exist in this new setup.

In essence, I think however Cider does it, we can do it similarly. It would probably be a good idea to look more into how they do it. :smiley:

bpringe commented 4 years ago

Also, I'm not sure if Cider requires this, but we could require that jack-in be run with the project file open, so there's no ambiguity about which project to start a repl for. If no file is open we could use the top-level directory project file, if one exists. Just some other ideas.

rayat-amperity commented 2 years ago

Any thoughts on this? Arguably mono-repos are a non-trivially popular codebase structure, and for any mono repo microservice based org ( eg where I work ), we need to often develop across services, thus also often necessitating multiple REPL connections. While I have started accepting opening silly numbers of VSCode windows, I can't help but wonder how emacs and others support multi projects, and what vsc could support.

If I were a more experienced clojurian I'd be happy to help more. I do have some experience building vscode extensions and LSP work, so in case I may be of help, please feel free to list any definite steps we could take in this direction!!

bpringe commented 2 years ago

I think the connection handling code in Calva maybe could use some reworking for supporting this, though I'm not sure much rework is needed.

We probably could store connections in state, let the user name them upon initially connecting (and maybe default the name to something reasonable), and then make it easy to switch between them.

Let's see what @PEZ thinks about what steps we could take toward this.

tekacs commented 2 years ago

To ask a perhaps nonsensical question -- it looks like the nrepl session is stored in state as clj, cljs or cljc?

https://github.com/BetterThanTomorrow/calva/blob/1962e126b59a76a35c9793982b743d2e048a4f35/src/nrepl/repl-session.ts#L5-L20

If that's the case, for the common CLJ+S case, could the connection sequence just ask for two ports, one for CLJ and one for CLJS -- and create two sessions, storing them at those two state values respectively -- and set CLJC to whichever is appropriate?

bpringe commented 2 years ago

@tekacs I think it somewhat works like that now in terms of storing and switching, but not in terms of how those connections are started or created.

My (current) thoughts on supporting multiple repl connections

@PEZ has done more work on this part of Calva than myself, so I may not understand all things we have to handle with connections, but here are my thoughts.

Say I have a folder open in VS Code that includes a backend project and a frontend project, each in their own folder, and as independent projects. The backend is a deps.edn project and the frontend is a shadow-cljs project. Let's ignore jack-in for now.

I could start a repl for each project in separate terminals, and connect Calva to each, one by one. When I do that, I could specify the host, port, repl type, and optionally give it a name and a root folder path (this path might be pre-filled depending on the currently open directory or an open file within it).

Connections could be stored as:

[
  {
    "name": "backend repl",
    "host": "localhost",
    "port": "3000",
    "type": "clj",
    "projectFilePath": "/Users/brandon/development/my-org/backend/deps.edn"
  },
  {
    "name": "frontend repl",
    "host": "localhost",
    "port": "3001",
    "type": "cljs",
    "projectFilePath": "/Users/brandon/development/my-org/frontend/deps.edn"
  }
]

I could have however many connections of each type that I want stored in Calva, and switch between them at will.

Furthermore, if I evaluate something in a file, Calva would automatically use the connection with a projectFilePath that's included in the current file's path. So, I could evaluate something inside of the backend project and it would automatically use the backend repl connection, and I could then evaluate something inside of the frontend project and it would automatically use the frontend repl connection.

tekacs commented 2 years ago

@tekacs I think it somewhat works like that now in terms of storing and switching, but not in terms of how those connections are started or created.

@bpringe that was the thinking -- as a 'quick solution' given that this is already how things are stored, I wondered if the current 'ask for a port' dialog could simply be replaced with... two 'ask for a port' dialogs in a row, one for CLJ, one for CLJS. This would only solve for the common case of one CLJ + one CLJS, but it might be a really easy addition to the current state of affairs, storage and behaviours?

bpringe commented 2 years ago

Let's see what @PEZ thinks.

could the connection sequence just ask for two ports, one for CLJ and one for CLJS -- and create two sessions, storing them at those two state values respectively -- and set CLJC to whichever is appropriate?

Calva already handles storing a cljs and clj connection, and handles switching between them, but this might only be the case for cljs REPLs that are started from a clj repl. I'm not sure the same mechanics would work with two standalone REPLs with the current code. I don't use this switching feature though, so I'll defer to @PEZ.

orestis commented 2 years ago

I have a slightly different use case for multiple nrepl connections.

We use a monorepo where most of the code is the “main” project but there’s some extra code for back office tools. These back office tools run in a separate environment where I often REPL into to do ad-hoc queries and such. But, often times the back office tools reuse code of the main project.

Anyway my desired workflow is to be able to manually connect to running REPLs from the same code folder. I don’t mind having to open a duplicate VSCode window to keep the sessions separate (I was doing this with Vim too). Some indication in the UI about which REPL is currently connected would be good.

As it works today, if I open a duplicate VSCode window and connect to a different REPL, then the calva.output file gets clobbered. I’m not even sure which REPL my commands go to.

I’m not sure if nRepl sessions have a unique ID, but in any case “namespacing” REPL artifacts like the calva.output window so that you can have multiple of those side by side would solve my issue (I think).

PEZ commented 2 years ago

I'm sorry for not having chimed in here. It's a great discussion! And it really is about time we solve this.

There are several use cases in people's minds. I think that @tekacs' suggestion is awesome and should be considered on its own, since afaiu it doesn't really demand more sessions, just offers more flexibility in how the two we do have are being initialized. If you're still around, @tekacs , please file a new issue about it. I'll do it in a few days if you don't seem to be around. 😄

@orestis use case is covered by #1313 I think. And I also think my last suggestion on that one is the way to go. Shouldn't be too hard to fix.

As for this particular issue, it looks to me like your idea should be explored, @bpringe. I'll have a think and might do some experiments and get back here.

bpringe commented 2 years ago

To expand on my idea above, I think the connections should be stored in an object and each connection should be keyed by some unique identifier, like the projectFilePath. This would allow for easier modification of the connections versus using an array.

orestis commented 2 years ago

The projectFilePath is probably not a good key though. Why not the nrepl host:port? Or even a unique identifier. Worth looking at what Conjure for nvim is doing…

bpringe commented 2 years ago

Yes, there's probably a better way to factor this, and exploring prior art is definitely wise.

davidyang commented 1 year ago

Hey - wanted to revive this thread and see if there's any progress or ideas that I might be able to start implementing/helping with.

bpringe commented 1 year ago

I don't think there's been much movement on this, if any.

I'll have a think and might do some experiments and get back here.

@PEZ Any new thoughts on this?

BetterThanTomorrow / calva

Multiple nrepl connections #76

My (current) thoughts on supporting multiple repl connections