Add support for multiple workspaces

[tengstrand]

Today it's possible to have multiple workspaces in a monorepo, and to share code between them, by using the :local/root syntax. However, the poly tool handles references to bricks like any library, and not like bricks. As a result, they are not included in the dependency calculation, and changes to a brick from another workspace will not trigger a test run + they are not listed in the info command as bricks, just as libraries.

To get this to work, we need to include all bricks and projects from all workspaces in all the calculations. Here is an idea of how that could be implemented. We could add a :full-name attribute to bricks and projects, that includes the alias of the workspace, e.g. myws/mybrick. An idea is to use namespaced keywords, like :myws/mybrick, which would make it easy to pick out both the name with (name full-name) and the namespace with (namespace full-name).

The workspace structure is already prepared to support multiple workspaces. The current workspace is stored in the :workspaces vector as a single element. When we add support for multiple workspaces, all workspaces can be stored here, and each of them can have an alias, which would be the workspace name (the name of the workspace directory) if not given.

For the different commands lik info, libs, and devs to work nicely, we could tell the tool which workspace is current. Let's say the current workspace is ws1 which refers to bricks in ws2. Then the the different commands, like theinfo command, would not prefix the ws1 workspace, but bricks in ws2 would show the prefix. The info command will show the current workspace, and a list of workspaces and their aliases, and maybe the relative path to the root of them.

At the root of the repository, we could have a polylith.edn config file, that specifies all the workspaces to include, and which one is current. It could look something like:

{:current-ws "w"
 :workspaces [:path ".", :alias "w"]
             [:path "shared", :alias "s"]

If the path is set to "." then the name of the repository will be used as workspace name (just as today).

If we s upport overriding settings, then we could specify the same workspace several times, but with different aliases. This is something that we may skip in the first iteration, but could be good to think about.

All bricks, projects, and profiles will be stored together in :bricks, :projects, and :profiles as today, but they will also include :ws-alias. If we use the alias instead of workspace name, it becomes possible to support that a workspace is used more than once, but with different settings.

When this issue is implemented, it will be a lot easier to add support for cljs.

[marksto]

Hi @tengstrand! Having this feature in Polylith will be awesome! I remember our discussion about this in Slack just a few months ago. The pace of recent developments is astonishing. Kudos to you, Sean, Furkan and Lee Read!

One quick question, if you don't mind. Could you, please, hint at how this sub-feature can be useful? What can it give?

it becomes possible to support that a workspace is used more than once, but with different settings

[tengstrand]

Thank you for the kind words @marksto!

it becomes possible to support that a workspace is used more than once, but with different settings

This is an idea I currently have, which would allow you to override e.g. test configuration settings. Another option is that each workspace only specify the workspaces they use, within their own workspace.edn config. That is maybe a simpler solution, but may introduce code duplication. I will experiment with the different alternatives, and see what works best!

[tengstrand]

My current idea is that each workspace.edn will have a :workspaces key where you can specify workspaces that you share code from, e.g.:

{:top-namespace "backend"
 :interface-ns "interface"
 :default-profile-name "default"
 :compact-views #{}
 :vcs {:name "git"
       :auto-add false}
 :tag-patterns {:stable "stable-*"
                :release "v[0-9]*"}
 :projects {"development" {:alias "dev"}}
 :workspaces [{:alias "s"
               :path "../shared"}]}

This means that every workspace will be self contained when it comes to configuration, and we don't need an omnipotent configuration file at the root of the workspace, as we discussed in ClojureScript support.

[marksto]

:workspaces [{:alias "s"
              :path "../shared"}]

@tengstrand, hmm, so using relative paths for wss effectively means they also can successfully end up not being nested? I.e. having the following one-level file structure:

<root-dir>
- <ws-a> <— can reference bricks from b and c
- <ws-b> <— can reference bricks from a and c
- <ws-c> <— can reference bricks from a and b

[tengstrand]

Yes, that's the idea.

[avocade]

👏🏻