Add Team support

[kristin-bagnall]

We're looking to combine the Github connector with https://hub.getdbt.com/fivetran/github_source/

Since all our repositories belong to a team, adding support for teams in the Fivetran connector would give the ability to create per team github analytics.

[fivetran-joemarkiewicz]

Currently evaluating adding Teams to the Github dbt packages and wanted to present a solution for this issue:

• Bring the Github team and repo_team tables into the source package.
• For the transform package, as the relationship between Teams and Repositories is M:1 a possible solution would be to create a string_agg field within the github_issues model which would provide all the teams associated with the repository.

Would the above solution work for how your company utilizes repositories and teams?

[martinmalek]

In our case the team repo relation is 1:1. I haven't tried adding multiple teams as owners of the same repo so I can't say if a N:1 relation is doable within github. That said if you solve it with a N:1 relation this should cover our 1:1 case as well.

[fivetran-joemarkiewicz]

Thanks for the response @martinmalek!

I just finished adding the team and repo_team tables to the source package, and added the teams aggregate field (repository_team_names) to the github__issue and github__pull_request models. This way you will be able to have team visibility in the final models! Would you be willing to install the mentioned working branch of the transform package and let me know if the results look accurate on your end?

Please be aware this working branch also includes renaming source and transform models to be consistent with our current naming standard. As such, if you install and run this working branch it will create new tables in your warehouse (for example github_issues has now been changed to github__issues and thus will create a new table when executed). If you are willing to test, you can change your packages.yml file to reference the following:

  - git: https://github.com/fivetran/dbt_github.git
    revision: joe-feature-updates
    warn-unpinned: false

Once you have updated your packages.yml file you may execute the below to to remove old package dependencies and install the working branch

dbt clean && dbt deps

Finally you can then execute dbt run.

Let me know if the results are what you expect and I can open this branch up for a PR to master.

Thanks again!

[martinmalek]

@fivetran-joemarkiewicz - I managed to install, run and find the repository_team_names in github__issue and github__pull_request. Though it doesn't seem that the teams made it into github__daily_metrics.

If you don't want to change the current behaviour of github__daily_metrics, what do you think about creating a github__team_daily_metrics and have descendants of that for week and month?

You can keep github__daily_metrics and it's descendants as is or rename them to github__total_daily_metricsor similar for extra clarity?

[fivetran-joemarkiewicz]

@martinmalek that is great news, I am glad the changes worked and you are able to see the team metrics in the github__issue and github__pull_request models!

Additionally, I agree that creating new metric models will be a good addition to this package. Unfortunately our team currently has a busy quarter ahead of us and will have to add the team metrics enhancement to our backlog. We can approach this enhancement in the future.

However, this package is open sourced and we highly encourage and accept collaboration! As such, you could easily fork the repository and add the team metric models then open a PR to merge your enhancements with the master github branch.