LIN-685, LIN-640 Pipeline Writer refactor

[andycui97]

Description

Refactor pieces around PipelineWriters. Changes:

• Split pipelinewriters into separate files. Base, Airflow and DVC get their own files.
• Update taskDefinition class. Instead of containing templated code in definition, it now contains all the info to fill the task jinja template, which is more robust and can be passed around and modified.
• Removed airflowCodeGenerator, instead opting to have Airflow specific functions be part of AirflowPipelineWriter.
• get_artifact_task_definitions and get_session_task_definitions moved from Airflow writer to base pipeline writer and return new taskdefinitions as defined above.
• Airflow codegen merged into one function _write_operators which handles templating of flavors.

Fixes LIN-685, LIN-640

Type of change

• [ X ] Refactor (non-breaking change which does not change functionality)
• [ X ] This change requires a documentation update

How Has This Been Tested?

Existing pytests.

[yoonspark]

Looks good overall. A couple of thoughts for further refactoring we can do to enhance contributor experience:

1. Currently, the "compositional" logic for DAG file is scattered between the framework-specific pipeline writer (e.g., AirflowPipelineWriter) and the Jinja template (e.g., airflow_dag_PythonOperator.jinja). We may consider moving as much compositional logic as possible into the Jinja template so that the contributor can focus on adapting/updating it. For instance, compositional logic in AirflowPipelineWriter.get_task_params_args() and tasks can be factored into Jinja template which may actually make it more clear as to what is being written. This migration of compositional logic to Jinja template may also make lighter and consistent the code for each framework-specific pipeline writer.

2. Current refactoring uses TaskDefinition as a contributor-facing abstraction, which hides unnecessary implementation details. I think we can expand this abstraction to even hide a lot of internal data structures still exposed in BasePipelineWriter. For instance, BasePipelineWriter's methods still use SessionArtifacts to extract information necessary for code generation. The proposal here is to instead use an expanded/augmented version of TaskDefinition so that such information extraction from internal data structures is not necessary; and BasePipelineWriter's methods simply use this contributor-friendly abstraction to carry on their compositional jobs.

Note that these are suggestions for future work, and they do not block merging of this PR.