When cross-building is enabled, the code is generated only for one version

[rabejens]

In my SBT project, my main Scala version is set to 2.11.8, but because I am also using Spark, I set up my project for cross-building for 2.10.6 and 2.11.8.

However, the +avro:generate-specific (the plus for crossbuild mode) puts the generated Scala code to target/scala-2.10/src_managed/main/compiled_avro. I manually have to copy it to the respective scala_2.11 folder.

Without the copying, a sbt +compile works, but when using the project in Eclipse, I manually have to add the 2.10 folder to my project, otherwise it shows a lot of compiler errors.

Is it possible to tell the plugin to output the generated code to both directories?

[julianpeeters]

I hadn't run into this before, thanks for the report. sbt-avrohugger defers to sbt here, so I'm not sure of quick fix (and my eta to look closer is probably on a "months" timeframe). PRs accepted, course. In the meantime, I wonder if you might have success by overriding scalaSource, so both Scala versions will (over)write to the same, non-targetdirectory?

[rabejens]

What should I put into the scalaSource override? How are the paths resolved?

[julianpeeters]

if you override with (scalaSource in avroConfig) := new java.io.File("src/scala/main/myscalaSource"), then the code will be generated into src/scala/main/myscalaSource instead of target/scala-2.10/src_managed/main/compiled_avro. (Come to think of it tho, if you have enums im your schema, I'm not sure avrohugger will be smart enough -- my have to a add a method to avrohugger that (or derives) a javaSource as well.

Thanks for poking at this. The source override feature is modified from sbt-avro, so my familiarity with it is not perfect, please try these suggestions and report more specific errors for quickest help.

[rabejens]

I got it to work with this setting:

(scalaSource in avroConfig) := {
      val base = thisProject.value.base
      new File(new File(new File(new File(new File(base, "target"), "scala"), "src_managed"), "main"), "compiled_avro")
    }

Why this "new File" clutter? Because this needs to build on Windows AND Unixes and the path format is different.

EDIT: Followup:

I finally got it to work both for command line and for Eclipse.

Here is how I did it:

  lazy val myAvroSettings = specificAvroSettings ++ Seq(
    (scalaSource in avroConfig) := {
      val base = thisProject.value.base
      new File(new File(new File(new File(new File(base, "target"), "scala"), "src_managed"), "main"), "compiled_avro")
    },
    managedSourceDirectories in Compile <++= baseDirectory { base =>
      Seq(
        base / "target/scala/src_managed/main/compiled_avro"
      )
    }
  )

Now I can add my myAvroSettings to a project where I need them:

lazy val some_avro_project = project.
  in(file("some_avro_project")).
  settings(myCommonSettings ++ myAvroSettings).
  settings(
    // more settings
  )

And the eclipse plugin also picks up the path correctly.

[julianpeeters]

Awesome, congrats and thanks for sharing!

[rabejens]

Hello, I am coming back to this one. For the newest version, I adapted this to the following:

  lazy val avroSettings = Seq(
                               sourceGenerators in Compile += (avroScalaGenerateSpecific in Compile).taskValue,
                               avroSpecificScalaSource in Compile := {
                                 val base = thisProject.value.base
                                 new File(new File(new File(new File(new File(base, "target"), "scala"), "src_managed"), "main"), "compiled_avro")
                               },
                               managedSourceDirectories in Compile ++= baseDirectory { base =>
                                 Seq(
                                      base / "target/scala/src_managed/main/compiled_avro"
                                    )
                               }.value
                             )

Then, I can add these settings with project.settings(avroSettings) as before.

[julianpeeters]

ok great, thanks for the update!

[sndnv]

We've been having this issue where sometimes the 2.12 compilation would run first and things would succeed (Scala 2.13 seems to be happy compiling with the 2.12 code) but the other way around things fail (2.12 is very unhappy about compiling with 2.13 code).

I tracked this down to the file caching done here - https://github.com/julianpeeters/sbt-avrohugger/blob/main/src/main/scala/sbtavrohugger/SbtAvrohugger.scala#L65

The files are cached when the first version finishes compiling and the other versions just don't bother since the cache says it's all good. I hope the fix is just using crossTarget instead (looks like it's work locally) - https://github.com/julianpeeters/sbt-avrohugger/pull/88

[julianpeeters]

https://github.com/julianpeeters/sbt-avrohugger/pull/88