search

apache / datafusion

Apache DataFusion SQL Query Engine
https://datafusion.apache.org/
Apache License 2.0
6.34k stars 1.2k forks source link

[DISCUSS] Use semantic PR titles to help with change log generation? #5601

Open andygrove opened 1 year ago

andygrove commented 1 year ago

We have been using github-changelog-generator [1] to generate changelogs for the Rust projects for some time now. It has served us well but is no longer workable, at least for DataFusion. This tool seems to pull down the entire project history using the GitHub API and we had to artificially slow it down to avoid hitting API rate limits, and it is now unusable due to the number of issues and PRs in this repo.

This weekend, I built a much simpler changelog generator in Python [2], that I am now using for the projects that I am the release manager for (datafusion, datafusion-python, ballista). It has almost the same functionality that we were getting from the previous generator, but takes less than a minute to run, compared to 30+ minutes for the old generator. It only hits the GitHub API for information about commits and pull requests in the release being documented, rather than accessing the entire project history.

I followed the same approach of using GitHub labels to categorize PRs (enhancements, bug fixes, docs, etc) but this requires a small amount of manual effort to add those labels and re-generate the changelog.

I noticed that some contributors are already prefixing PR titles with "feat:", "feature:", "fix:", "docs:", etc. I plan on updating the changelog generator to recognize these prefixes as well, to help automate my job.

I wonder if it is worth formalizing these "semantic titles" more, and maybe having CI enforce them. It would improve the quality of our changelogs and reduce the burden on release managers.

I would appreciate any feedback on this idea.

Thanks,

Andy.

[1] https://github.com/github-changelog-generator/github-changelog-generator [2] https://github.com/andygrove/changelog-genie

andygrove commented 1 year ago

Mailing list discussion: https://lists.apache.org/thread/9mhk203nvrs17w3dvt4m6xtpzv342nj7

andygrove commented 1 year ago

TIL this is called Conventional Commits:

https://www.conventionalcommits.org/en/v1.0.0/

alamb commented 1 year ago

I think the idea behind conventional commits is a good one, and it helps with changelog generation.

However, like all human entered data, it is often messy / noisy and a simple bot that enforces conventional commit PR titles will not ensure the content is valuable in those commits.

andygrove commented 1 year ago

Perhaps we can automatically add labels based on the PR title, and the release manager can remove the labels as needed

andygrove commented 1 year ago

I also just realized that the new changelog generator is not yet looking at the labels on the issues that the PRs close, and that will help a lot once I implement that. I can also show the list of issues that were closed by PRs.