ploomber / soopervisor

☁️ Export Ploomber pipelines to Kubernetes (Argo), Airflow, AWS Batch, SLURM, and Kubeflow.
https://soopervisor.readthedocs.io
Apache License 2.0
45 stars 18 forks source link

Templating tutorials #37

Closed edublancas closed 9 months ago

edublancas commented 2 years ago

Our tutorials are currently in .rst but have some limitations:

  1. Hardcoded branch: tutorials have a git clone https://github.com/ploomber/soopervisor command that users need to run. However, for testing purposes, we'd like that command to be dependent on the current git branch, otherwise, if we are in branch dev, our test will still pull the code in master
  2. Development args: for dev purposes, we pass PLOOMBER_STATS_ENABLED=false so our dev runs do not appear on posthog, however, showing this flag in the public tutorial is confusing

We can solve this by using jinja (similar to what we've done in the projects repo): have a jinja template that compiles two version: the public one (pulls from master, does not have the env var) and the internal one (pulls from the current branch, has the env var)

Note that this also applies to the tutorials Dockerfiles since we want them to use the github version when developing but the stable one for users.