"Downstream" build triggering

[tjtelan]

This pattern is particularly prevalent:

Build+Deploy Library repos -> Build+Deploy Service Where the Service pulls in newer libraries that are in development.

It would be convenient if either/or a Library could trigger a downstream build or a Service could watch upstream build.

---

As a secondary goal: We should consider what would need to be done to display the build-related relationships/ordering of a full microservice architecture.

[shankj3]

Service "watching" would be the best here
I propose we add a new block to the ocelot.yml that would manage this functionality

Example for a mock level11consulting/ocelot:

image: docker.metaverse.l11.com/ocelot/base:latest
buildTool: go
subscriptions:
- acctRepo: level11consulting/pogo 
  branches:
    - master
    - dev 
  branchesToTrigger: 
    - master
    - rc/.*

This block would be stored in a new Postgres table, active_subscriptions that would be updated and maintained only by the master branch (as of now)

subscribed_acct_repo	subscribing_acct_repo	subscribed_branches	branches_to_queue	id
level11consulting/pogo	level11consulting/ocelot	{master,dev}	{master,rc/,*}	1
level11consulting/pogo	level11consulting/gopro	{master}	{master,rc/,*}	2

for this example, if pogo is built successfully, this table will be checked to see if there are any new builds to put on the queue. in this case, two repositories are signed up for downstream triggers. if the pogo branch is master, then both repos will be built for all of their branches_to_queue, or master and all rc/.* branches will be built at their latest commit.

If the build branch is dev, only level11consulting/ocelot will be built.
If any other branch of pogo is built, then nothing will be triggered downstream.

---

This table will be updated every time the subscriptions block is changed on master. The entry in the active_subscriptions table will be deleted if it is deleted from the ocelot.yml on the master branch. To keep track of what is triggering what, build_summary will also get a new column of triggered_by. This will be set at first entrance of the build to the table (the AddSumStart function).
Triggered by can be set to :

• manual trigger
• poll
• push_event
• pr_event
• subscription_event

[tjtelan]

I can't think of anything obviously bad with this strategy. Downstream jobs will become responsible for documenting their dependencies.

One concern I have subscription loop detection in the Ocelot pipeline may be required, as right now I've only really considered straight pipelines. I can imagine some awful feedback loop that floods werker queues by accident.

---

I think that there should be some context that gets passed to the triggered downstream job, such as:

• Ocelot Environment variables, such as GIT_BRANCH, GIT_HASH and GIT_HASH_SHORT
• Custom Ocelot.yml environment variables

I think there is a slippery slope here with longer pipelines that should be kept in mind. I'd really like to keep build context passing as simple as it needs to be.

[shankj3]

other use case: a singular engine or codebase being deployed to multiple places. that deployment / configuration should not be in the other codebase, it should be in its own wrapper codebase that subscribes to events off the parent codebase building

[shankj3]

branch mapping to mitigate builds going off willy nilly

[mariannefeng]

Similar to how in Trident, all services pass a correlation ID downstream (that can then be later traced), would be cool if when an upstream project is triggered, it creates a new ID for every downstream dependency. That way, people can query for the state of an "id", and get back a list of all builds who are a part of that id, and see the state of every build FOR that id.

Also would be neat if the ID was easily distinguishable from the random guids created for builds (perhaps prefix of some sort or a different random id generating algorithm)

[tjtelan]

We still need this. We should redesign the config file representation and all of the workflow re: master branch. I still think there is some loop detection that will trip us up. Going to hide the past thinking.

[tjtelan]

I started doing some of the design work for this.

A question I want to be able to answer is: What repos in the org are (explicitly) dependent on Repo X?

I also want the trigger relationship to be defined in the orb.yml. Because of the question above, I need to keep some state in the DB so I can start builds an "upstream" repo is finishing.

There needs to be some attention paid to preventing circular dependencies (at least a single level deep) so we're not perpetually building.

---

Using Orbital as an example, its config might look loosely like this:

  stages:
  [...]
  trigger_on:
    repos:
      - name: git-url-parse 
        watch-branch: main (default)
        action:
          - notify
          - build orbital>main ENV=somethung
          - build orbital>dev
      - name: git-event
        watch-branch: all
        action: build orbital>all
      - name: git-meta

Orbital would build upon the successful builds of these 3 other repos. I imagine that this would neatly fit into the build state machine in the finishing state. We would check in the DB for any triggers defined on this repo and start a build with the JobTrigger type: SUBSCRIBE_TRIGGER (current name)

Additionally, we can specify branches we'd want to trigger on, and also what kind of actions to take. Including sending notifications prior to starting builds, starting multiple builds for different branches, and starting builds with environment variables.

There are a few default behaviors that I also want to include, that will need documenting. Like the git-meta case above, where I am not very specific about watch-branch or action.

One of the bigger details I'll expand now being how changes to this trigger_on block changes behavior in Orbital.

I propose that when we add in repos, that by default, Orbital sets a canonical_branch (using the Git repo's default, unless specified or changed). When the trigger_on block changes in the canonical branch, Orbital will modify the triggers table, but otherwise no changes.

Same behavior if we change the canonical branch through Orbital.

While I've been thinking about it longer, I should point out in the documentation that this is DB-only state with a default value. (I can't think of a story where it adds clarity more than confusion if canonical_branch were in the orb.yml. How would you change the value?)

---

The DB might look like this:

The triggers table is new. I would have a row for each repo+branch combination for which there is a trigger. The repo_id_list column would have the name of a map of other repo+branch combinations to start builds.

Lastly, last_triggered_by would include the build_target id that "pulled the trigger". I am hoping that I can use this info specifically to catch some naive loops but I need to work out more of the details.

[tjtelan]

I will need this functionality in order to monitor changes to the orb config in canonical_branch.

https://github.com/tjtelan/git-meta-rs/issues/4