Example Bazel Monorepo
Note: Currently supporting the latest Bazel version as at mid June 2021, 4.1.0
Example Bazel-ified monorepo, supporting Golang, Java, Python, Scala, and Typescript.
Cloud Infrastructure-as-Code is done using Terraform.
I use this project to explore how Bazel works with different languages and developer tools, and keep a record of best-practices I've learnt. So it is a work in progress. Others can use it to check out the Bazel way of doing things and use parts as a reference implementation.
Rather than the typical To-Do list, this project's code uses the contrived scenario of a book shop and reading catalogue website called Antilibrary. 📗📕📒📚
Getting Started
Prerequisites:
- Install Bazel (Currently supporting ~=
4.x.x) - Python 2 or 3. Should only be required to do some bootstrapping under-the-hood.
yarnornpmfor the NodeJS and Typescript code
Bazel aims to be 'build anything, anywhere' system, so building and testing should be as simple as bazel test //.... If it's not, please create an issue.
Why use a Monorepo?
The following few articles together provide a good overview of the motivations behind maintaining a Monorepo. For heaps more information, korfuri/awesome-monorepo is a good place to go.
- Why Google Stores Billions of Lines in a Single Repository
- Monorepos, please do!, by Adam Jacob, former CTO of Chef
- Repo Style Wars: Mono vs. Multi, by Peter Seibel
- Advantages of Monorepos, by Dan Luu
Related Projects
- github.com/lucperkins/colossus - A demo using Bazel in monorepo fashion. Compared with this project, it goes far deeper on microservice architecture components and Kubernetes, and is not focused on Bazel.
- github.com/enginoid/monorepo-base - Employs Bazel, gRPC, and Kubernetes like the above, and is similarly not as broad and deep on Bazel as this project.
Project Structure
Golang Support
There's Golang code in /cli. It implements a simple CLI for the common 'Blind Date With a 📖' product.
Dependency Management
Third-party dependencies are managed in 3rdparty/go_workspace.bzl.
Java Support
There's a Spring Boot (with PostGres) application in /store-api and some other Java code in /store/layoutsolver.
Dependency Management
Its third-party dependencies are managed by rules_jvm_external in the WORKSPACE (See the # JAVA SUPPORT section).
Scala Support
There's Scala code contained in scala-book-sorting.
Dependency Management
Its third-party dependencies are managed
by johnynek/bazel-deps. The usage of that tool is wrapped up in a script
as tools/update_jvm_dependencies.sh.
To use it, you update tools/dependencies/jvm_dependencies.yaml and then run the script.
Python Support
There's Python code in the /book_sorting and /scraping.
bazelbuild/rules_python is used for the core py_* rules.
Dependency Management
In order to add new third-party packages for Python, add them to 3rdparty/requirements.in and run bazel run //3rdparty:requirements.update.
Gradual Type-Checking (MyPy)
thundergolfer/bazel-mypy-integration is used to check any type annotations at bazel build time.
Infrastructure-as-Code (Hashicorp Terraform)
The infrastructure/ top-level folder contains Terraform defining various AWS resources and their configuration.
Development
Build
bazel build //...
Testing
bazel test //...
Continuous Integration (CI)
This repository's CI is managed by Buildkite, the CI platform used by Pinterest and Canva to manage Bazel monorepos, as well as being used by the Bazel open-source project itself.
Deployment & Distribution
Deployable artifacts are pushed to S3 under commit-hash-versioned keys.
Currently only the store-api deploy/fat JAR is deployable.
graknlabs/bazel-distribution is used to publish Python packages to PyPi.
Build Observability + Analysis
This project is using Buildbuddy.IO. Every build run locally or in CI get its own https://app.buildbuddy.io/invocation/xyz123... URL which analyses and records the build's information.
Linting
thundergolfer/bazel-linting-system is used. ./tools/linting/lint.sh will lint all source-code in the repo and ./tools/linting/lint_bzl_files.sh will lint all Bazel files.