feat: workshop improvements

[kirederik]

After running the workshop a few times in recent weeks, I found a few places where we could improve/simplify the learner's journey.

Using Compound Promises:

Too much happens at once

• Current
  • We start by explaining that we need to register the platform as a worker. We then install the compound promise, and nothing happens. We then label the destination and watch the system reconcile.
• Problem
  • It's confusing. A lot is happening on Compound Promises already, and seeing a "failure" does not make it clearer. Learners have to grasp, all at once, 1) the platform becoming a worker, 2) the promise depending on promises and 3) the destination labels.
• Improvement
  • A better alternative is to make it "just work". For that, I propose the following changes:
  • Move the "platform as a worker" setup (including the label) to the Prerequisites (i.e., the section before Part I even starts) so when the learner starts the Compound Promise section, there's no setup needed.
  • In "Using Compound Promises", make it so that it just works. Apply the Promise, see the dependencies showing up, send a request, and get it all working.
  • Once the user sees the working system, explain what's going on:
    • bring in the pipeline outputting to the platform cluster via labels
    • bring in the sub-promises pipeline outputting to the worker cluster via labels

Outdated compound promise

• Current
  • The compound promise we use declares the subpromises as dependencies.
• Problem
  • We moved away from this pattern, and we now lead people towards requiredPromises. This is the Promise we write later on. This brings confusion to learners since they are unsure of what pattern to follow.
• Improvement
  • Make use of requiredPromises in the compound promise.

Writing a Promise

Inline dependencies

• Current
  • We write a Promise with inline dependencies just to extract them to a pipeline in the next section
• Problem
  • Again, two patterns. The learner "understands" something just to see it being done differently later. Inline dependencies are also not the pattern we "recommend".
• Improvement
  • Split the section into two (this could all be a single page or maybe two separate pages):
  • The "Promise Bootstrap" stage
    • define the API, add the dependencies pipeline, add the resource pipeline (straight on the CLI, no customisation), install the promise and see the dependency showing up on the worker. Send a resource request and see the "Hello from Pipeline" log.
  • The "Resource Request" stage
    • Update the Resource pipeline to create the resources

Improving the Workflows

Unclear goal, complex section

• Current
  • In this section, we introduce the following concepts today:
  • Pipeline design principles
  • Promise Dependencies as pipelines
  • A "test" framework for the promise
• Problem
  • Lots of boilerplate copy-and-paste. The "test framework" really complicates the entire page (lots to copy and paste, complex scripts we don't explain). The "test" doesn't actually "fail" -- the learner has to check the files themselves, but what are they looking for exactly? The update of the promise is a no-op (we are just moving dependencies to a pipeline)
• Improvement
  • Remove this section entirely? Move the TDD / Design Principles into a section of the Kratix documentation

Surfacing information via status

createdAt adds little value

• Current
  • We add two fields to the state: createdAt and message. createdAt touches on the need to make status idempotent.
• Problem
  • It's confusing and makes this section quite long. We have already touched on idempotency in other sections, there's no need to bang on the same key again.
• Improvement
  • Remove createdAt, focus on surfacing the app url to users via status.
  • Add a "Bonus challenge" around dealing with idempotency in the status field.

Across sections

Can't run the pipeline script locally

• Current
  • Leaner can only test the pipeline script with a complicated docker build/run combo
• Problem
  • Unnecessary complication
• Improvement
  • Make the pipeline scripts locally executable: instead of docker build + docker run, we can just rewrite the scripts so they can be executed locally. To test, you just need to run the script (we can provide the input/output/metadata dir as env vars or arguments or assume test if /kratix does not exist).

[shano]

1. On using Compound Promises - 100% agree, getting it spun up and the reasoning about the system as it sets itself up seems like a more natural flow, can we capture the commands explicitly to "see" the system progressing/working as expected.
2. On writing Compound Promises - 100% agree, when I went through the workshop I felt the same confusion. It felt arbitrary I'm being shown two ways to do a thing., it then "felt" but wasn't explicitly called out that pipelines are the recommended way to do things as it's much more flexible but showing me multiple options while I'm still trying to understand everything seems strange.
3. Improving Workflows - It would be good to know the objective of these docs, it is to get into a platform engineering loop? So make changes, deploy changes, see the changes in action? If so, there's perhaps a simple example to get a person quickly iterating on changes to get a feel for what this process is like.
4. Surfacing info - Agreed, it's long for what it's trying to convey. This feels beyond an intro workshop and are more later stage optimisations engineers would naturally want to do and would hit docs.

Overall, very much in favour of these changes!

[richcooper95]

Using Compound Promises:

Too much happens at once

Agree with the changes to explain the underlying flow after showing things working. I think we have a nice diagram somewhere (maybe in the workshop slides) for how this flow looks?

Outdated compound promise

Agree that requiredPromises does simplify the workshop by only showing one way of doing things, but just to call out that we'll need to also touch on Promise Releases earlier in the workshop if we go for this approach (since requiredPromises are versioned).

Writing a Promise

Inline dependencies

Agree that we should only add dependencies via Pipelines. Though the fact that we're not mentioning the dependencies key at all in the workshop does bring up the age-old question of whether we should remove the dependencies key altogether from Promises. If we're not removing it now, maybe we could just have a callout saying "for simple cases/small dependencies, you can use the dependencies key..."?

Improving the Workflows

Unclear goal, complex section

I've seen this section be well-received in workshops, so I'm reluctant to remove it altogether. That said, we could specify the goal more clearly (e.g. demonstrating composability and testability of Pipeline container images), and perhaps also introduce a real "test" to ./scripts/test-pipeline where we check the dependencies exist?

(I also like the idea of moving/copying some info about workflow best practices to the Kratix docs!)

Surfacing information via status

createdAt adds little value

I think we could keep this too, since it touches on (a) a concrete example of idempotency, and (b) the ability to read the status from the /kratix/input/object.yaml. It's a quick section and IMO it's useful to concretely demo these things!

Across sections

Can't run the pipeline script locally

I like this goal, though I'm not quite sure what you mean on implementation!

[kirederik]

@shano [on Improving Workflows ] It would be good to know the objective of these docs, it is to get into a platform engineering loop? So make changes, deploy changes, see the changes in action? If so, there's perhaps a simple example to get a person quickly iterating on changes to get a feel for what this process is like. @richcooper95 [on Improving the Workflows] I've seen this section be well-received in workshops, so I'm reluctant to remove it altogether.

Yeah, see the comment above

We originally wanted to introduce the "TDDing your Promise" concept, but it got mixed with moving the dependencies into the pipeline at the same time. Maybe moving the deps to a pipeline directly will clear this up of this "second thing we are trying to achieve" and makes it more on point...

@richcooper95 [on surfacing information] I think we could keep this too, since it touches on (a) a concrete example of idempotency, and (b) the ability to read the status from the /kratix/input/object.yaml. It's a quick section and IMO it's useful to concretely demo these things!

Yeah, there's value in the "reading from status" but, to me, it feels like a secondary concern to "surfacing information" (hence my suggestion of adding it as an extra challenge).

@richcooper95 [on running pipelines locally] I like this goal, though I'm not quite sure what you mean on implementation!

You can do things like:

#!/usr/bin/env bash

input_object="${1:/kratix/input/object.yaml}"
output_dir="${2:/kratix/output}"

name="$(yq .metadata.name "${input_object}")"

kubectl create deployment "${name}" \
  --image nginx \
  --dry-run=client -o yaml > ${output_dir}/deployment.yaml

So now you can run locally by:

$> ./pipeline.sh test-object.yaml /tmp/

[ChunyiLyu]

Briefly picked this up end of Oct 17. Small WIP that moves the "platform as a worker" setup (including the label) to the part-0/02-create-clusters pushed to https://github.com/syntasso/kratix-docs/pull/133

[kirederik]

closing this one since it's marked as completed 👀