ploomber / soorgeon

Convert monolithic Jupyter notebooks 📙 into maintainable Ploomber pipelines. 📊
https://ploomber.io
Apache License 2.0
78 stars 20 forks source link

soorgeon adding a section prefix to each task #41

Open idomic opened 2 years ago

idomic commented 2 years ago

Ideally when converting this pipeline the section (which is repeating therefore redundant) shouldn't appear there, or at least the prefix can be parameterized.


- source: tasks/section-1-load-data.py
  product:
    y: output/section-1-load-data-y.pkl
    nb: output/section-1-load-data.ipynb

- source: tasks/section-2-preprocess-data.py
  product:
    y: output/section-2-preprocess-data-y.pkl
    nb: output/section-2-preprocess-data.ipynb

- source: tasks/section-3-visualize-data.py
  product:
    nb: output/section-3-visualize-data.ipynb```
edublancas commented 2 years ago

yeah, I did that as an easy way to ensure that the products and tasks are unique, since ploomber does not allow duplicates. but it'd be better to only add the prefix when needed (e.g. when there are two sections or products with the same name)