folder_path and model_prefix functionality for generate_model_yaml

[erkanncelen]

Describe the feature

I want to introduce folder_path and model_prefix functionality for generate_model_yaml macro.

This way the macro can be used as:

• {{ generate_model_yaml(modelprefix='fct') }} --> all models starting with fct_
• {{ generate_model_yaml(folder_path='marts/marketing') }} --> all models in marts/marketing folder
• {{ generate_model_yaml(folder_path='marts/marketing', modelprefix='fct') }} --> all models in marts/marketing folder starting with fct_

Please let me know if you think this'd be a nice addition to the codegen so that I can finalise and submit my PR!

Describe alternatives you've considered

The alternative to this approach would be providing manually made lists to the macro, which is already possible. It is however time consuming and repetitive. I believe this approach will make dbt folks' lives easier.

Additional context

This feature is not database specific, at least not in my initial design. I basically utilise manifest.json information, to retrieve all models in a project, with model names and folder paths.

Who will this benefit?

These features will be extremely beneficial when we start documenting our dbt projects. Combining the two, we will be able to generate YAMLs for any group of models we desire. I personally used my approach at two of my clients already.

Are you interested in contributing this feature?

Yes I am! I actually have the PR almost ready, just missing the tests, waiting for the first approval.

[erkanncelen]

Hi @joellabes , do you think this'd be a nice addition to the codegen? If so, I want to polish it up and finish the PR

[joellabes]

Oooooh this is a really good idea! Sorry for the delay in getting back to you.

Instead of overloading the generate_model_yaml macro, what about a new macro called something like get_models_by_prefix_in_directory(prefix, directory) which returns an array of model names that can be passed directly into the existing unchanged macro?

[erkanncelen]

Hi @joellabes , thanks for your input. Your suggestion really makes sense.

I created a macro named get_models in macros/helpers directory. It takes directory and prefix arguments. Based on these arguments, it returns an array of matching model_names retrieved from the graph object.

Here I completed the PR with all requirements checked: https://github.com/dbt-labs/dbt-codegen/pull/115 I created an integration test for "model_" prefix, which returns an array of 4 models. The test passes in all 4 environments.

Please let me know if this is good to go!