Make include/exclude paths processing faster

h3adache commented 2 months ago

Is there an existing issue for this?

[X] I have searched the existing issues

Description of the bug

adding include/exclude paths results in a huge slow down.

Steps To Reproduce

run git-cliff in any repo with a large commit range (>= 100 commits)
observe that performance should be good.
add an include-path
observe a large slowdown.

I observed a huge slowdown generating a changelog for a large commit range and was able to track down the slowness to this block https://github.com/orhun/git-cliff/blob/main/git-cliff-core/src/repo.rs#L63-L91. removing the --include-path option confirmed the cause.

Expected behavior

Don't be slow? :)

Screenshots / Logs

No response

Software information

Repeatable on osx m1 and linux. does not seem to be os specific.

Additional context

No response

welcome[bot] commented 2 months ago

Thanks for opening your first issue at git-cliff! Be sure to follow the issue template! ⛰️

orhun commented 1 month ago

Thanks for the report! I'm guessing it is due to making a lot of git2 calls and matching the glob against the list of files that are changed for each commit. I'm thinking if rayon (parallelizing) might help with regards to making things faster.

h3adache commented 1 month ago

Right that's one way to do it. I wonder if the library that is used to get the git commits already has a way to handle include/exclude paths using git's own built in capability

exclude

e.g.

git log --oneline -20 -- ':(top)config/**' ':(top)website/**' ':(exclude,top)website/blog/**'

orhun commented 1 month ago

It is probably a good idea to check out git2 to see if it is supported: https://github.com/search?q=repo%3Arust-lang%2Fgit2-rs%20exclude&type=code

orhun / git-cliff