In https://github.com/ccao-data/data-architecture/pull/626 we split up the export_models() function in dbt/scripts/utils/export.py to support a new requirement that the export_qc_town_close script filter the result set of each model query differently. The design works, but it's a bit clunky, and I imagine it'll be hard to extend. Dan suggests an iterative improvement:
Kind of a bizarre idea, but maybe something like:
Have a query for each model
Have a filter for each model
Create a unique set of all the query values across models, run them, then populate the models using the results (and apply the filter to each one).
Might be more complicated? Maybe save it for the next incremental improvement.
I think this is probably a better direction for the export model to go overall. It also strikes me that the save_model_to_workbook function logic could be owned by the ModelForExport class, since it mostly depends on attributes of that class. Similarly, I think a potential query/filter interface might make sense to move onto the ModelForExport object too.
In https://github.com/ccao-data/data-architecture/pull/626 we split up the
export_models()function indbt/scripts/utils/export.pyto support a new requirement that theexport_qc_town_closescript filter the result set of each model query differently. The design works, but it's a bit clunky, and I imagine it'll be hard to extend. Dan suggests an iterative improvement:I think this is probably a better direction for the
exportmodel to go overall. It also strikes me that thesave_model_to_workbookfunction logic could be owned by theModelForExportclass, since it mostly depends on attributes of that class. Similarly, I think a potentialquery/filterinterface might make sense to move onto theModelForExportobject too.