Bazel for Eta

[Jyothsnasrinivas]

Description

It would be great to have Bazel support for Eta and will help Eta's adoption in large startups and companies who tend to follow monorepo model.

This issue is to track Bazel support for Eta and to discuss how to overcome any issues.

Tagging @jin and @johnynek who have both shown interest in writing the bazel rules.

A starting point of the discussion is: https://github.com/typelead/eta/issues/853

[johnynek]

@jin do you want to make a google doc and/or publish your WIP code?

I have followed rules_haskell fairly closely. There it seemed to require quite a bit of knowledge about how ghc-pkg worked in order to implement the rules. They didn't just call ghc.

To me, ideally, eta would have enough CLI flags that we can use it directly, since having etlas and bazel both in play make it complex to deal with the fact that etlas expects to own state directories, but bazel also expects to own all the state.

In addition to hopefully only using eta, it would be great if we can bootstrap the eta from source, or in the worst case only fetch jars and run eta on the jvm.

finally, it would be great to look at: https://github.com/FormationAI/hazel to see if we can use that to generate bazel dependencies for haskell packages that we can build with eta.

A doc, with some design goals would be great to help us split this work up.

I think a good v0 would be the ability to compile eta code locally and depend on other bazel java rules (java, scala) to use java and scala libraries.

a v1 should be able to consume haskell/eta libraries.

[puffnfresh]

@johnynek you can substitute ghc-pkg with eta-pkg

For Nix:

• I configure the package.conf.d with all the Eta dependencies
• Eta and Etlas do create files under $HOME - I made Nix set HOME to a specific directory during builds
• I do use Etlas, because that uses the project's build information
• I put a a script called coursier on the PATH which just lists the Maven jars

I don't know Bazel pretty much at all but it sounds like the same strategy might work.

https://github.com/eta-lang/eta-nix/blob/etaPackages/eta-modules/generic-builder.nix

[jin]

@johnynek I got java_binary.runtime_deps <- eta_library to work. PTAL: https://github.com/jin/rules_eta

[jin]

I've proposed a bunch features in the README as a start. It also contains the Java/Eta sandwich.

In addition to hopefully only using eta, it would be great if we can bootstrap the eta from source, or in the worst case only fetch jars and run eta on the jvm.

It took me over 15 minutes to compile eta on a Macbook Pro, so I don't recommend this. It'll be great if we have a direct distribution of the eta binary that we can pull with a repository_rule.

[jin]

Eta and Etlas do create files under $HOME - I made Nix set HOME to a specific directory during builds

I believe this is why I had to pass the Bazel flag --action_env=HOME to make it work. If not, eta will get this error:

$ bazel build //examples/basic:main
..
ERROR: Compiling main-eta using Eta failed (Exit 1)
eta: eta: panic! (the 'impossible' happened)
  (Eta version 0.8.6b2):
    v_unsafeGlobalDynFlags: not initialised

Please report this as a Eta bug: http://github.com/typelead/eta/issues

Target //examples/basic:main failed to build
..

@Jyothsnasrinivas can eta be made to not depend on the $HOME env var?

[Jyothsnasrinivas]

@jin Great work on getting a small example working already!

It took me over 15 minutes to compile eta on a Macbook Pro, so I don't recommend this. It'll be great if we have a direct distribution of the eta binary that we can pull with a repository_rule.

Here are the links to the binaries:

• Linux
• Mac OS
• Windows

You can substitute the last part of the links with eta-pkg and eta-serv.jar for the respective binaries.

Please note that Windows is an exception and non-JAR binaries have a .exe suffix.

@Jyothsnasrinivas can eta be made to not depend on the $HOME env var?

Yes that can be done. We can add another flag to configure the .eta directory.

[Jyothsnasrinivas]

@jin Just added -homedir flag. If you run eta -homedir somdir Main.hs it will expect that somedir/package.conf.d/ is a valid package database that has all the boot-libraries.

We distribute binary versions of all the boot packages in every release. And they are downloaded and installed using Etlas. You are probably better off using Etlas instead of Eta because you'll end up duplicating the same work Etlas is doing.

[jin]

Thanks! That makes it significantly simpler as we can use Bazel's non-hermetic repository rules to bootstrap the required directories within Bazel's execution root for Eta to function, and pass those locations into the rules.

I haven't really looked deeply at how Etlas work, but from my initial experience I'm echoing @johnynek's thoughts:

To me, ideally, eta would have enough CLI flags that we can use it directly, since having etlas and bazel both in play make it complex to deal with the fact that etlas expects to own state directories, but bazel also expects to own all the state.

So far, I've been trying to implement the simpler logic with primitive calls as much as possible without using a more powerful tool like Etlas. It helps to keep things simple and we can easily track the granularity of decomposed actions using eta_library targets. From my experience so far, I think Etlas will be used to manage the dependencies during the (pre-)loading phase, and Eta for the actual granular compilation.

[Jyothsnasrinivas]

Sure that sounds good!