search

ezbz / gitlabber

Gitlabber - clones or pulls entire groups tree from gitlab
MIT License
472 stars 78 forks source link

Clone/pull by a group id #86

Closed konfetka1989 closed 1 month ago

konfetka1989 commented 2 years ago

If you have access to a large amount of groups, it may take much time to load an entire tree. It would be nice to have an opportunity to specify an exact group id which you want to clone.

I propose my solution: https://github.com/konfetka1989/gitlabber/commit/e48c267c351ea7e591d4d502e52ec123563eea03

Please consider it. Thanks.

I didn't create a pull request, because CONTRIBUTING.md states I have to discuss my changes with the owner first.

ezbz commented 2 years ago

thanks for submitting code and suggesting this feature, I do realize for large setups reading the entire tree structure can take a long time and incur a lot of API calls.

2 concerns we need to address with this feature:

  1. I am assuming it plays well with globre patterns (if a user specifies a globre pattern that doesn't match with the selected group it will result in an empty tree but that is to be expected I guess)
  2. You chose to use group id for root filtering but for the majority of gitlabber usage people get accustomed to use project and group names/paths. Should this be considered in the root filtering feature? looking forward this may translate into more issues being opened for this.
konfetka1989 commented 2 years ago

Hi @ezbz

Thank you for the feedback! Now I see several issues with my approach. I think I need to revisit this feature.

konfetka1989 commented 2 years ago

Hi @ezbz

I created a draft pull request which provides a new approach to using some group as a root: https://github.com/ezbz/gitlabber/pull/89

Could you please review it? Thank you.

radoslav006 commented 2 years ago

Hi everybody! It will be very helpful for me as unfortunately I am hitting gitlab rate limits, so - sadly - I cannot use this tool. AFAIU this merge request should limit the number of http requests.

ezbz commented 1 month ago

Hi everybody! It will be very helpful for me as unfortunately I am hitting gitlab rate limits, so - sadly - I cannot use this tool. AFAIU this merge request should limit the number of http requests.

Python gitlab implements rate limiting and obeys gitlab's throttling, see: https://python-gitlab.readthedocs.io/en/latest/api-usage-advanced.html#rate-limits