rules_scala or rules_scala_annex?

kojustin commented 5 years ago

The README says this project is for use with rules_scala but the play-routes-compiler/BUILD.bazel file uses rules_scala_annex. Which set of Scala build rules are expected to work?

jjudd commented 5 years ago

AFAIK both rule sets will work. We currently use it internally at Lucid with rules_scala_annex, but have used it in the past with rules_scala. Provided the rules you are using accept srcjars in srcs`, then you should be good to go.

kojustin commented 5 years ago

Ok cool.

I was experimenting with these rules trying to build a simple play application. I thought the issue I was having was related to the differences between rules_scala and rules_scala_annex but perhaps it is not. The error I am getting is...

$ bazel build //testsvc:play-routes
ERROR: Analysis of target '//testsvc:play-routes' failed; build aborted: error loading package '@io_bazel_rules_play_routes//play-routes-compiler': Extension file not found. Unable to load package for '@rules_scala_annex//rules:scala.bzl': The repository could not be resolved
INFO: Elapsed time: 0.072s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
    currently loading: @bazel_tools//tools/zip ... (2 packages)

My BUILD file for this package

load("@io_bazel_rules_play_routes//play-routes:play-routes.bzl", "play_routes")
load("@io_bazel_rules_twirl//twirl:twirl.bzl", "twirl_templates")

# Twirl templates
twirl_templates(
    name = "twirl-templates",
    srcs = glob(["app/**/*.scala.html"]),
    include_play_imports = True,
    source_directory = "app",
)

# Play routes
play_routes(
    name = "play-routes",
    srcs = ["conf/routes"],
    generate_reverse_router = True,
    include_play_imports = True,
)

and the Scala part of my WORKSPACE file looks like this

#
# Basically, Scala + Play Framework build rules
#

rules_scala_version = "a89d44f7ef67d93dedfc9888630f48d7723516f7"  # update this as needed

http_archive(
    name = "io_bazel_rules_scala",
    strip_prefix = "rules_scala-%s" % rules_scala_version,
    type = "zip",
    url = "https://github.com/bazelbuild/rules_scala/archive/%s.zip" % rules_scala_version,
)

load("@io_bazel_rules_scala//scala:scala.bzl", "scala_repositories")

scala_repositories()

load("@io_bazel_rules_scala//scala:toolchains.bzl", "scala_register_toolchains")

scala_register_toolchains()

# https://github.com/lucidsoftware/rules_play_routes
rules_play_routes_version = "c6970c79ad493bb21649e21a0d956773d3562ed6"

http_archive(
    name = "io_bazel_rules_play_routes",
    strip_prefix = "rules_play_routes-%s" % rules_play_routes_version,
    type = "zip",
    url = "https://github.com/lucidsoftware/rules_play_routes/archive/%s.zip" % rules_play_routes_version,
)

# https://github.com/lucidsoftware/rules_twirl
rules_twirl_version = "8ee464dc4840a9b7dd3182a391553f710722d7ab"

http_archive(
    name = "io_bazel_rules_twirl",
    strip_prefix = "rules_twirl-%s" % rules_twirl_version,
    type = "zip",
    url = "https://github.com/lucidsoftware/rules_twirl/archive/%s.zip" % rules_twirl_version,
)

Looking through this repository I'm not sure how the play-routes-compiler is supposed to be used. It does not look like it is declared as a Bazel rule (e.g. the XXX = rule(...)) so is this error expected?

jjudd commented 5 years ago

Ah. I believe there is an undocumented dependency on annex. Your application does not need to use annex, but the twirl and play routes compilers use it to build the binary they use to compile things.

If you include annex in your workspace file, I imagine this will work. https://github.com/andyscott/rules_scala_annex#usage

Looks like we need to update the doc for both play routes and twirl. We'll do that. Probably wouldn't hurt as well to have a project that shows an example of these working end to end.

thundergolfer commented 5 years ago

Even after including annex in the workspace file, I get the following when trying to use rules_play_routes:

 bazel run //ingestion/apps-flyer
ERROR: /private/var/tmp/_bazel_jbelotti/a8a1f81a1422a72c79c6c81641a8c816/external/io_bazel_rules_play_routes/play-routes-compiler/BUILD.bazel:3:1: no such target '@scala//:scala-library': target 'scala-library' not declared in package '' defined by /private/var/tmp/_bazel_jbelotti/a8a1f81a1422a72c79c6c81641a8c816/external/scala/BUILD and referenced by '@io_bazel_rules_play_routes//play-routes-compiler:play-routes-compiler'
ERROR: /private/var/tmp/_bazel_jbelotti/a8a1f81a1422a72c79c6c81641a8c816/external/io_bazel_rules_play_routes/play-routes-compiler/BUILD.bazel:3:1: no such target '@scala//:scala-parser-combinators': target 'scala-parser-combinators' not declared in package '' defined by /private/var/tmp/_bazel_jbelotti/a8a1f81a1422a72c79c6c81641a8c816/external/scala/BUILD and referenced by '@io_bazel_rules_play_routes//play-routes-compiler:play-routes-compiler'
ERROR: /private/var/tmp/_bazel_jbelotti/a8a1f81a1422a72c79c6c81641a8c816/external/io_bazel_rules_play_routes/play-routes-compiler/BUILD.bazel:3:1: no such target '@scala//:scala-compiler': target 'scala-compiler' not declared in package '' defined by /private/var/tmp/_bazel_jbelotti/a8a1f81a1422a72c79c6c81641a8c816/external/scala/BUILD and referenced by '@io_bazel_rules_play_routes//play-routes-compiler:play-routes-compiler'
ERROR: /private/var/tmp/_bazel_jbelotti/a8a1f81a1422a72c79c6c81641a8c816/external/io_bazel_rules_play_routes/play-routes-compiler/BUILD.bazel:3:1: no such target '@scala//:scala-reflect': target 'scala-reflect' not declared in package '' defined by /private/var/tmp/_bazel_jbelotti/a8a1f81a1422a72c79c6c81641a8c816/external/scala/BUILD and referenced by '@io_bazel_rules_play_routes//play-routes-compiler:play-routes-compiler'
ERROR: /private/var/tmp/_bazel_jbelotti/a8a1f81a1422a72c79c6c81641a8c816/external/io_bazel_rules_play_routes/play-routes-compiler/BUILD.bazel:3:1: no such target '@scala//:scala-xml': target 'scala-xml' not declared in package '' (did you mean 'scala'?) defined by /private/var/tmp/_bazel_jbelotti/a8a1f81a1422a72c79c6c81641a8c816/external/scala/BUILD and referenced by '@io_bazel_rules_play_routes//play-routes-compiler:play-routes-compiler'
ERROR: Analysis of target '//ingestion/apps-flyer:apps-flyer' failed; build aborted: Analysis failed

thundergolfer commented 5 years ago

@kojustin Did you get this working with rules_scala?

kojustin commented 5 years ago

@thundergolfer, no i haven't gotten this to work.

thundergolfer commented 5 years ago

@kojustin I've got a fork going over here as I am pretty confident that this does not work with the standard rules_scala, which is a show-stopper.

My fork is refactoring it to work with rules_scala, and generally improving how this project handles its 3rd-party dependencies.

thundergolfer commented 5 years ago

https://github.com/thundergolfer/rules_play_routes/commits/master Is now in a working state in our Bazel repo that uses rules_scala.

kojustin commented 5 years ago

After investigating this some more I figured out a way to use these rules without having to pull in all the dependencies of this repository. I've opened up https://github.com/lucidsoftware/rules_play_routes/pull/7, this is also based off of https://github.com/thundergolfer/rules_play_routes/commits/master.

The main idea to use a Bazel repository_rule to pull down the routes compiler as a deploy JAR. This technique allows us to break the dependency chain, which is necessary until Bazel implements a solution for recursive workspaces, https://github.com/bazelbuild/proposals/blob/master/designs/2018-11-07-design-recursive-workspaces.md

@jjudd is this an approach you guys are interested in?

jjudd commented 5 years ago

It's been a few months and I'm coming back to this. Sorry it took so long. I want to fix this for both rules_play_routes and rules_twirl. My current plan is to

Move the command line tool into its own GitHub repo/workspace to make versioning simple.
Build and publish the command line tool to Maven from that repo. Cross publish for various versions of Play/Scala.
Move the tests into their own WORKSPACE in this repo.

That should remove the dependency on any particular version of the Scala rules from this repo. This is similar to what you did in #7.

I'm thinking of using deploy/fat jar for the command line tool, creating a jar for all active versions of Play on maven, and then taking in a string in the repository rule when loading the rules. Alternative is to make a skinny jar and require people to pass in a jar. Anyone have a preference?

Should hopefully be able to get it done in the next couple week in the evenings.

cc @borkaehw @SrodriguezO for their thoughts.

SrodriguezO commented 5 years ago

Just realizing I forgot to comment.. but we discussed with James offline. This sounds great.

Here's my WIP: https://github.com/lucidsoftware/rules_play_routes/pull/22

lucidsoftware / rules_play_routes

rules_scala or rules_scala_annex? #5