search

MeltanoLabs / tap-gitlab

Singer.io Tap for extracting data from Gitlab's API
GNU Affero General Public License v3.0
8 stars 25 forks source link

add files to merge requests and add line count - [opened] #31

Closed pnadolny13 closed 2 years ago

pnadolny13 commented 2 years ago

In GitLab by @xiaozhou.wang on Jun 17, 2021, 11:27

Merges add-merge-request-files-and-files-changes -> master

Pulls files changed by each merge request and calculate a line count for each file.

Resolves #43

pnadolny13 commented 2 years ago

In GitLab by @xiaozhou.wang on Jun 17, 2021, 11:27

changed the description

pnadolny13 commented 2 years ago

In GitLab by @tayloramurphy on Jun 21, 2021, 16:53

assigned to @aaronsteers

pnadolny13 commented 2 years ago

In GitLab by @aaronsteers on Jun 23, 2021, 15:32

Commented on tap_gitlab/init.py line 26

Hi, @xiaozhou.wang. If I understand correctly that this is a net-new stream, what are your thoughts on having the separate fetch_merge_request_files config toggle versus using standard Singer selection metadata?

In general, the way we'd implement this would be to simply add the stream directly and check at runtime if it is selected or not.

I can see some exception perhaps if the performance is very bad and we don't want to "break" existing pipelines - but even then I haven't seen this pattern before. Can you say more about your thoughts on this new stream and any risks of adding it without a toggle flag, performance expectations, etc.?

Thanks!

pnadolny13 commented 2 years ago

In GitLab by @aaronsteers on Jun 23, 2021, 15:33

Commented on tap_gitlab/init.py line 26

(Just to be clear, I do see that this tap has fetch_merge_request_commits as a similar toggle, but I'm not familiar with the background and considerations that went into it.)