very concise way to specify dependencies

[cvogt]

Many projects, especially early only need a bunch of dependencies, nothing else.

It have been requested from several people (e.g. @densh) to be able to specify those in a more concise way than a fully blown Scala file. Scala has the necessary infrastructure to easily support something like this now.

How about we support a file

dependencies.cbt and a file plugins.cbt

in the root directory of your build. They both have the format:

org.scala-lang:scala-reflect:2.11.8
com.chusaai::shapeless:1.2.3
...

Single : for java dependency, :: for scala dependency.

CBT would integrate these things in a way that does not prevent users from writing Scala builds as well, e.g. by reading these things into actual CBT Dependency objects in super class.

One could extend this concept and also support defining settings, variable substitution e.g.

// settings
scalaVersion=2.11.8
scala.target=$projectDirectory/target

org.scala-lang:scala-reflect:$scalaVersion
com.chusaai::shapeless:1.2.3
...

This would make the internal implementation quite a bit more complicated, but it's on the table. It would probably have to generate a mixin trait that CBT mixes in by default and users can too.

How this is integrated into a CBT build should be very easy to explain and very well documented, so that it is not magical for people.

The big question is where to draw the line. This is a slippery slope sliding down into creating a new language with similar but slightly different semantics than Scala and slightly briefer syntax. I think the options are: 1. Avoid it entirely and suck up the boiler plate, it's not what makes things hard for people anyways and codegen can help. 2. Support it, but with clear limitations where people have to move to Scala. Make it very very clear how this maps to/integrates with the Scala builds. 3. Support it without limitation and risk a hard to understand rabbit hole.

Any comments on this?

Any comments on the whole thing?

@densh @jvican @olafurpg ?

(Please add other interested people you know of to this issue)

[cvogt]

cc @nafg @darkfrog26

[darkfrog26]

What is the minimum necessary code to add a dependency now?

[cvogt]

import cbt._
class Build( val context: Context ) extends BaseBuild{
  override def dependencies = super.dependencies ++ Resolver( mavenCentral ).bind(
    MavenDependency("org.scala-lang","scala-reflect",scalaVersion),
    ScalaDependency("com.chusaai", "shapeless", "1.2.3")
  )
}

(actually "com.chusaai" %% "shapeless" % "1.2.3" works too instead of ScalaDependency("com.chusaai", "shapeless", "1.2.3") for sbt compat)

[cvogt]

However cbt tools createBuild will create this file for you and you only have to fill in the concrete libs you need.

[darkfrog26]

My philosophy is that if it's so complicated that it needs a tool to generate the code, then it's too complicated.

Providing convenience functionality to cut that down both in lines and complexity would be my vote. I have a philosophical problem with the .sbt files and that's essentially what you'd be moving toward. Scala is a powerful language and is fully capable of representing this cleanly and effectively.

[olafurpg]

I'm 👍 for listing common configuration data (ints, booleans, strings, list[string]) like library dependencies, scalac options (probably a few other, too) outside of .scala files. I think Scala is particularly bad for this since 1. re-compiling code just to add a new dependency/option is slow and 2. Scala syntax for argument lists requires quotes around strings and commas to separate each argument. I recommend you opt for a standard configuration language like hocon instead of rolling your own syntax.

[olafurpg]

There's a wide range of excellent libraries to parse hocon into case classes. Pureconfig seems to be most popular today, Ficus is another one. I use my own metaconfig.

[nafg]

HOCON makes a lot of sense. Maybe a better file naming convention would be .cbt/dependencies.conf

Also I don't see why the Scala syntax couldn't be shortened, e.g., to mvn"group:name:version"

[cvogt]

I hear you @darkfrog26, I tend into the same direction, but I keep hearing requests for a more concise format and want to be open minded. E.g. I could see us generally optimizing in the direction you suggest, but potentially offering a limited, more concise alternative for when the boiler plate is particularly painful, e.g. smallish libraries that just have a few dependencies and settings, nothing else, hence the proposal.

@olafurpg

1. re-compiling code just to add a new dependency/option is slow

I understand where you are coming from but in my experience with CBT this is not a problem. It's 1 or 2 seconds total one-time overhead, much faster than reload in sbt.

@olafurpg / @nafg

2. Scala syntax for argument lists requires quotes around strings and commas to separate each argument

Yep, the reason that other languages can be more concise than Scala in the detail is the reason for this proposal. While also recognizing, that Scala's power is very helpful for many use cases, but common ones exist where it is not needed. So let's consider a syntax optimization for these s simple and common ones. I believe that for anything beyond these, custom syntax will road will lead you to maven or sbt equivalents (may they be hocon instead of xml) that make anything beyond the simplest things hard to understand, do or maintain.

I recommend you opt for a standard configuration language like hocon instead of rolling your own syntax. HOCON makes a lot of sense. Maybe a better file naming convention would be .cbt/dependencies.conf

How would my initial examples look in HOCON?

mvn"group:name:version"

not a bad idea. I think we could get to something like this in CBT out of the box

import cbt._
trait Build extends Dependencies(
  mvn"org.scala-lang:scala-reflect:$scalaVersion",
  scala"com.chusaai::shapeless:1.2.3"
){
  // settings
  override def scalaVersion=2.11.8
  override def scalac = super.scalac.copy( target = projectDirectory / "target" )
}

[olafurpg]

CBT this is not a problem. It's 1 or 2 seconds total one-time overhead, much faster than reload in sbt.

God, I really should give cbt another try. Reload takes ~25 seconds in the scala.meta build. Do you use zinc to re-build the build?

How would my initial examples look in HOCON?

libraries = [
  org.scala-lang:scala-reflect:2.12.1
  com.chusaai::shapeless:1.2.3
]

Notably, there is no need for commas or quotes.

extends Dependencies(mvn"org.scala-lang:scala-reflect:$scalaVersion", ...

That looks kinda nice, esp. with trailing commas. The mvn interpolator could probably easily be turned into a macro to get compile-time validation.

[cvogt]

God, I really should give cbt another try.

I am working towards 1.0. The code organization suffered over the last year (but the intrinsic structure is simple). Will cleanup for 1.0. The Milestone is 80% cleanup tasks :).

Do you use zinc to re-build the build?

Absolutely :). Plus CBT caches the classloaders, so only the classloader of that build class has to be reloaded, everything else stays in memory.

(Side note: I'll work towards 1.0 in the traditional form. This ticket is a consideration for usability improvements in 1.1)

[darkfrog26]

What if the build file were as concise as the following?

import cbt._

class MyBuild extends Build {
  dependencies ++= Seq(
    "org.scala-lang" % "scala-reflect" % scalaVersion,
    "com.chusaai" %% "shapeless" % "1.2.3"
  )
}

It's pretty close to the example @cvogt already gave, but with some of the boilerplate removed. I can't imagine it being difficult to remember how to do this from scratch and you're only adding a couple of additional lines from the proposed .cbt style. The major benefit of something like this though is that as the complexity of your application grows you don't have to switch to another style, you just add to what you have. This is what I hate about SBT. The .scala files were always far more useful for very complex projects, but now they've entirely deprecated that. If you provide one solid style and make it as simple as possible for getting started while being as flexible as possible for the complex scenarios then you only need one solution and it becomes far easier to learn.

[cvogt]

@darkfrog26 I have thought about basically providing well defined syntactic sugar on top of Scala. I don't think mutation is on the table. Instead it would have to be clear, local and simple rules how sugar desugars into declarative, immutable Scala. Maybe not exactly the syntax you proposed, but something similar that e.g. nicely desugars to the necessary override def. Ideally the syntax would be invalid Scala which the language extension makes valid for that purpose, so we don't prevent valid Scala code.

Maybe override foo ++ = Seq(...) desugars to override def foo = super.foo ++ Seq( ... ), but then what if you want Seq(...) ++ super.foo?

Ideally we'd want something we try to actually get integrated into Scala itself, by trying to make a case for it's use by offering it in CBT. Challenges for in the mean time would be getting IDE support, but I have confidence at least in the IntelliJ guys to be open to that.

[darkfrog26]

@cvogt though I'm not against finding an immutable path to offer a concise solution, I dislike the idea of introducing more complexity into the build by introducing non-Scala syntax. For what little gains we make in conciseness, we add exponentially in the learning curve for new users. I would much rather just retain the complete override def than see something like that. If the Scala language does introduce it that's a completely different story, but until that time I don't think new proposed language constructs belong anywhere but in proposals.