Request: Would be nice to have a way to define vars in dependencies.yaml

[davidsantiago]

It just happens that the largish project I work on has many dependencies that have related jars, but which don't work nicely as modules, but we'd like to keep the versions matching. I worry about forgetting to update a version on one of them. It would be nice if dependencies.yaml had a section in options to define vars, which you can then reference using some syntax in the keys for version.

I'm like 10 years rusty on Scala, but would a PR for something like this be welcome?

[johnynek]

I would love a way to do this. How can we?

We are just using raw yaml here. Does it have a way?

An alternative would be allow a few different configuration formats. For instance hocon, a superset or JSON (and maybe yaml?) allows supstitutions: https://github.com/lightbend/config/blob/master/HOCON.md

Alternatively we could imagine a skylark/star lark API if we could factor skylark out of bazel (there is a go implementation but not jvm outside of bazel that I know of).

Or maybe we can link to the bazel implementation since clearly we require bazel for this tool.

[davidsantiago]

Oh, being able to write the deps in starlark would be amazing. But when I looked into that just now, it doesn't seem like that is a very proximate prospect, as I think you're aware.

My thinking was actually much more limited. The dependencies.yaml file would have a new key in the options map:

options:
  buildHeader: [ "load(\"@io_bazel_rules_scala//scala:scala_import.bzl\", \"scala_import\")" ]
  languages: [ "java", "scala:2.11.8" ]
  resolverType: "coursier"
  resolvers:
    - id: "mavencentral"
      type: "default"
      url: https://repo.maven.apache.org/maven2/
  strictVisibility: true
  transitivity: runtime_deps
  versionConflictPolicy: highest
  vars:
    dropwizardVersion: 1.3.5
    jacksonVersion: 2.9.6
    etc: 1.2.3

Then in the map of dependencies, you could reference them with some escaping syntax in the version keys, like

com.fasterxml.jackson.core:
  jackson:
     lang: java
     version: $jacksonVersion
     modules: [core, databind]
com.fasterxml.jackson.datatype:
  jackson-datatype:
     lang: java
     version: $jacksonVersion
     modules: [jsr310, jdk8]

[johnynek]

Actually, yeah, this seems reasonable.

Happy to accept a PR. Might bikeshed the vars name a bit. Maybe constants or versionConstants on only do this expansion in version. If we have to expand all strings (language, modules, etc...) it might be a bit complex. Are there other use cases you can imagine other than setting shared versions?

[davidsantiago]

Totally cool with that. My natural tendency to generalize led me to think about where else this feature might be useful, but I could not come up with anything other than sharing versions for related jars. So my intention was to just start with that.

[ceason]

Can't this be done with YAML anchors?

com.fasterxml.jackson.core:
  jackson:
     lang: java
     version: &jacksonVersion "2.9.6"
     modules: [core, databind]
com.fasterxml.jackson.datatype:
  jackson-datatype:
     lang: java
     version: *jacksonVersion
     modules: [jsr310, jdk8]

Right now this doesn't work (maybe the currently used YAML library doesn't support it?), but this is normally how I "DRY" semantically equivalent values in YAML.

I wonder if the YAML parser has an option for anchors, or if another parser with support could be dropped-in in a non disruptive way.

[johnynek]

This seems to indicate there are some hoops to jump through with Jackson and ymal: https://stackoverflow.com/questions/40074700/jackson-yaml-support-for-anchors-and-references and if I recall we are using Circe with Jackson for yaml (on the phone at the moment)

Might need to update Circe and/or file an issue there.

Cc @travisbrown

[rdnetto4]

Can't this be done with YAML anchors?

My 2 cents is that we'd be better off having an explicit vars/properties/whatever section, rather than using YAML anchors. The reasons for this are:

• it provides a cleaner separation between the definition of the properties and their usages
• it's easier for other tools to consume those values from the YAML file (whereas if we used anchors, they'd have to look them up by the coordinates of a given artifact, rather than just by the property name)
• it simplifies migration from maven, which uses the <properties> section to solve the same problem