search

LineaLabs / lineapy

Move fast from data science prototype to pipeline. Capture, analyze, and transform messy notebooks into data pipelines with just two lines of code.
https://lineapy.org
Apache License 2.0
661 stars 58 forks source link

[LIN-696] Add pipeline integration tutorial #843

Closed andycui97 closed 1 year ago

andycui97 commented 1 year ago

Description

Create a step-by-step tutorial for integrating a new pipeline framework

Fixes: LIN-696

Type of change

yoonspark commented 1 year ago

Overall feedback:

  1. The reader might be confused to get references to technical details in the concepts section. It might be helpful to have more clear delineation/separation between materials for conceptual understanding vs. concrete implementation steps. It may help to have explicit sections for each -- to help the user see where one starts and another begins.
  2. (Stretch, Optional) It may help to have some simplified visualization of how the discussed concepts relate to one another.
yoonspark commented 1 year ago

@andycui97 Based on our previous conversation, my understanding is that we want to incorporate feedback from Thomas and Lazar in to this PR before merging it. My sense is that there's no urgent need to merge this into main as they are the only two relevant reader for now.

andycui97 commented 1 year ago

Overall feedback:

  1. The reader might be confused to get references to technical details in the concepts section. It might be helpful to have more clear delineation/separation between materials for conceptual understanding vs. concrete implementation steps. It may help to have explicit sections for each -- to help the user see where one starts and another begins.
  2. (Stretch, Optional) It may help to have some simplified visualization of how the discussed concepts relate to one another.

For 1, I'll add a sentence at the top that will let them jump to the checklist.

Going to skip 2, not sure what type of diagram might help here, we can sync up if you have something specific and clear in mind.

yoonspark commented 1 year ago

Left one more content feedback + a bunch of grammar feedback. Two overarching style suggestions:

  1. LineaPy is referenced as "Lineapy", "lineapy", and "Linea". Standardize the reference into "LineaPy".
  2. Unify all references of task definition ("task definition", "TaskDefinition", etc.) into TaskDefinition to help the reader understand that we are talking about the same new construct in codebase.
andycui97 commented 1 year ago

2. task definition

sgtm will do