Refactor API for proper sub-project generation flexibility

[sirthias]

The basic idea behind the plugin is great: provide a "generation" layer on top of the SBT project/module that allows for convenient, programmatic generation of sub-projects/modules along a number of axes.

However, it feels like the current API falls short of providing the user with actual and proper access to the generative power of the underlying approach. The only way in which the user can actually configure a project based of the actual point in the n-dimensional matrix is "inside" the definition of individual settings/tasks, via the virtualAxes key.

Things that I'd like to do for certain points along the matrix axes but not for others (which are clunky or even impossible with the current API):

• enable/disable plugins (e.g. only for Scala 3 + JS)
• add inter-project dependencies (e.g. depend on another module only for the Scala 2 incarnation of the module, not Scala 3)
• add settings/tasks (i.e. not add that settings at all for certain matrix points)
• change the file system location of the project (e.g. have an entirely different folder for the Scala 3 variant of the same module)
• define "holes" in the matrix

All of this would be easy with an API that, for example, allows the user to supply a function (Matrix.Point, Project) => Option[Project] along with helpers that make it easier to have things like .dependsOn(...) point to the "right" (TM) target module.

Essentially, the current API provides a limited framework, which exposes the power of the basic approach only for certain, relatively simple uses cases. It would be great to turn this around and provide a toolbox, which provides helpers to make hard things easier but doesn't artificially restrict generative power.

[sirthias]

A first fix could to be to provide another overload of customRow along these lines:

def customRow(
  autoScalaLibrary: Boolean,
  axisValues: Seq[VirtualAxis],
  process: (Seq[VirtualAxis], Project) => Project
): ProjectMatrix

The idea is that within the process function a user has access to the axes configuration that the function is transforming.

[keynmol]

I tend to agree - we've worked around it in Weaver by providing syntax extensions to the ProjectMatrix, i.e. this one

Along with .full and .sparse(withScala3=..., withJS = ... for the same reasons. I'm interested in having it generalised.

Adding this more powerful version of customRow is not difficult, but I think it'd mean that whatever virtual axes you'll get access to will be sensitive to where that customRow is with regards to the full definition of the project matrix?

[eed3si9n]

enable/disable plugins (e.g. only for Scala 3 + JS)

The return type of ProjectFinder#apply is Project:

https://github.com/sbt/sbt-projectmatrix/blob/cd7f4453a63d3201ea2a2e7665c7bd793aec10f0/src/main/scala/sbt/internal/ProjectMatrix.scala#L137

So once you get to it, I am guessing you can do whatever you want with it like:

lazy val core12 = core.jvm("2.12.13")
  .enablePlugins(....)
  .settings(....)

[sirthias]

@eed3si9n it seems to me that the core value proposition of sbt-projectmatrix is to allow me to say

lazy val core = (projectMatrix in file("core"))
   .settings(...)
   ...

and have all incarnations of the core module be described/contained in that one val. Then later I can say core.projectRefs and refer to all of them at once, e.g. for aggregation.

If I have to split out individual module incarnations into their own vals I'm kind of back to where I came from. Then sbt-projectmatrix doesn't really add much value anymore. For example, I couldn't use core.projectRefs anymore, or could I?

[sirthias]

@keynmol Interesting! Thank you for the pointer!

[eed3si9n]

For example, I couldn't use core.projectRefs anymore, or could I?

ProjectRefs just contains the internal URL of the subproject so it should just work even if you split out a few of the exceptional subprojects into lazy vals. Assuming it works, I feel like that gives best of both worlds (matrix and subproject) without re-learning everything.