Add experimental macro support to Zinc 1.0

[jvican]

This is a meta issue to discuss the details of new experimental macro support in Zinc 1.0.

I think that the fact that Zinc 1.0 does not handle macro incremental compilation is a huge blocker for both the community and macro authors, who are more than we probably estimate. I've hit this limitation several times during work time and seeing all my project be recompiled is quite frustrating.

I'd like to have experimental macro support in Zinc enabled under a flag. Zinc will not promise continuous development of this feature, as this feature has been developed by external OSS committers @Duhemm and @xeno-by and macro dependency analysis relies on Macrotracker. It's experimental because we won't maintain Macrotracker.

There were previous attempts to merge this into sbt 0.12.x/0.13.x:

• https://github.com/sbt/sbt/pull/1757
• https://github.com/sbt/sbt/pull/1778
• https://github.com/sbt/sbt/pull/1836

I am porting these changes to Zinc 1.0. I've also been reviewing the PRs and the quality of the work is astonishing. There will be some minor changes in the way it's implemented in Zinc 1.0, partly because the codebase has changed since we got the contributions, but the meat of the solution stays.

There are several strategies we can do:

• Enable macro incremental compilation when incrementalMacros is true. In that case, build tools are forced to pass in the macrotracker compiler plugin in the extra field of ScalaInstance. Otherwise, a warning will be emitted.
• Enable macro incremental compilation when build tools pass in the macrotracker compiler plugin. As this plugin is only meant to be used for incremental compilation, its mere presence could be used to detect whether the user wants incremental macros or not.

The experimental macro option, if added, will go to IncOptions.

The benefit of this approach is that we will reuse all these options when Scala Meta is definitely stable and Scala Macros are deprecated. Macro incremental compilation will be always outsourced to a compiler plugin, so Meta maintainers will be able to provide such plugin and benefit from automatic incremental compilation, since the semantics of the dependency analysis will be the same.

At the time of this writing, I've already ported https://github.com/sbt/sbt/pull/1778 and I'm on my way to do https://github.com/sbt/sbt/pull/1836. I'll leave https://github.com/sbt/sbt/pull/1757 for another moment since I think it's not that critical. If someone is interested in it, I'll gladly accept a PR.

/cc @xeno-by @Duhemm

[romanowski]

I don't know if we need another issues for this but we should also consider at least a hook (or any other way) to capture changes done by compiler plugins (since I know a case where compiler plugin is a project dependency).