Some functionality has been duplicated in the pipeline scripts. In particular,
[ ] Parquet transcoding functions have been copied several times
[ ] SQL template rendering exists in several places
Refactor this functionality into a a module and explore moving it to the lib/ directory. Search for other duplicated functionality and refactor it as well.
Some functionality has been duplicated in the pipeline scripts. In particular,
Refactor this functionality into a a module and explore moving it to the
lib/
directory. Search for other duplicated functionality and refactor it as well.