Issue: Retrieving list of commits

[Aetherinox]

At present, I'm looking to utilize commitMode instead of getting pull requests.

When I switch the configuration over to commitMode, the script outputs nothing. Even though there are 5 or so commits.

🚀 Load commit history
  ⚠️ Executing experimental commit mode
  ℹ️ Comparing - '1.0.2...1.0.3'
  ℹ️ Found 5 commits from the GitHub API
  ℹ️ Retrieved 5 commits

📦 Build changelog
  ℹ️ Sorted all pull requests ascending: {"order":"ASC","on_property":"mergedAt"}
  ℹ️ Used 0 transformers to adjust message
  ✒️ Wrote messages for 5 pull requests
  ℹ️ Ordered all pull requests into 3 categories
  ✒️ Wrote 0 categorized pull requests down
  ✒️ Wrote 5 non categorized pull requests down
  ✒️ Wrote 0 ignored pull requests down
  ℹ️ Filled template

However, in the output I simply see

Run echo "Printing Variables"
Printing Variables

---- CHANGELOG ---------------------------------------------------------------

---- CHANGELOG ---------------------------------------------------------------

The changelog should be in the middle, and when I post it to a release, it's blank.

If I remove commitMode, it at least posts "No changes" to my release.

Wondering if I'm just missing something.

  - name: Build Changelog
    id: changelog
    uses: mikepenz/release-changelog-builder-action@v3
    with:
      commitMode: true
    env:
      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

and I'm using ${{ steps.changelog.outputs.changelog }}

[Aetherinox]

As a side note, if I use a json template, I get the following:

 <details>
<summary>Uncategorized</summary>

- asdasd
   - PR: #0
- dasd
   - PR: #0
- dasd
   - PR: #0
- dasd
   - PR: #0
- fix(test): something
   - PR: #0

</details>

But they all list

• PR: #0

Am I going to have to create a special Json template to specifically only handle a list of commits? I'd also need to see if there's a config to get rid of the blank PR being listed below.

[Aetherinox]

Edit. Alright, I've sort of figured this out. The template (json) needed to be tweaked a bit.

Sorry about that, just needed to invest a little time.

---

Also, in case anyone wants to utilize the pattern to follow the Git Conventional Commit Standards, patterns are slightly different

    "label_extractor": [
        {
        "pattern": "^(build|ci|chore|doc|docs|wiki|remove|deprecate|security|dependency|package|feat|feature|fix|bug|perf|optimize|refactor|revert|style|test):(.*)",
        "target": "$1"
        },
        {
        "pattern": "^(build|ci|chore|doc|docs|wiki|remove|deprecate|security|dependency|package|feat|feature|fix|bug|perf|optimize|refactor|revert|style|test){1}(\\([\\w\\-\\.]+\\))?(!)?:(.*)",
        "target": "$1"
        }
    ],

[mikepenz]

Good day @Aetherinox

I see you already found some answers to your question.

A few things ahead. I see you use @v3 -> I'd advice to use @v4 as this one gets all the latest updates and has some updates to the default template (for v4 for example would show uncategorized items by default)

I assume you also identified the configuration to adjust the default pr_template to not include the PR #?

Also thank you very much for providing your label extractor definition!

Do you have some other outstanding questions?

[Aetherinox]

Good catch, I didn't even realize I was using v3.

And yes, I figured out the PR # question, I went ahead and made my own configuration json, added the pattern matching, and enabled commits.

The only thing I thought would be cool would be the possibility of a PR + Commit mixture.

I've tested for a few hours, and I see in PR mode, it completely ignores all commits and only focuses on pull requests. But if I flip it to commit mode, then it strictly categorizes commits based on the label and ignores commits the PR functionality.

Unless I'm missing something I haven't tested yet.

As for v4, I'll update. Kind of funny I even missed that, I've been on this repo and in my code for a few hours, should have caught that.

[mikepenz]

@Aetherinox great to hear. I think the problem was that a sample in the readme still used @v3 opened a PR to rework the README a bit, and updated those.

Also added a sample with commit mode and your label extractor regex to the README for others to have an easier time to get started.

At this time this is not possible. Do you have some more details on the usecase for this?

[Aetherinox]

Oh nice, the examples will help. I had to go digging around and do a bunch of tests to figure out if I was in the right direction lol.

Yeah I figured the PR + Commit combo would be a challenge, but it's fine. Theoretically I should be migrating to a PR-based structure anyway.

For now, I just added a boolean in my workflow script so that I can flip between a commit or PR based release.

The only usecase that came to mind was if I decide that on a new version, I complete say three PR requess, which would appear in the change-log for the release, however, I then have 4 or 5 individual commits that weren't associated with a PR. Because I often find myself doing that.

Then the changelog script could categorize the pulls into their normal categories like they do now with the labels, but also list the individual commits.

[Aetherinox]

Oh, one more thing and then I'm done, just as a future update (unless it's possible now). In commit mode, everything goes into its correct category, ie: Features, Bugs, etc.

Maybe some type of way to reverse the pattern so that it will categorize properly like it does now, but when it actually places that item in the list, it removes the beginning of the string. Because it looks sort of odd to have a list like this:

Features

• feat: added something
• feat: something else new
• feat: something else.

Whereas, stripping out the front word, the result would be:

Features

• added something
• something else new
• something else

So the front word would be removed, but go into the correct category.

And then in the workflow config

with:
  fromTag: "${{ env.TAG_LAST }}"
  toTag: ${{ github.ref }}
  configuration: ".github/changelog-configuration.json"
  stripCategory: true           <-----------------------

That's just something else thrown out there, but the rest works perfect. Makes my life a hell of a lot easier now.

[mikepenz]

Yeah I figured the PR + Commit combo would be a challenge, but it's fine. Theoretically I should be migrating to a PR-based structure anyway.

For now, I just added a boolean in my workflow script so that I can flip between a commit or PR based release.

The only usecase that came to mind was if I decide that on a new version, I complete say three PR requess, which would appear in the change-log for the release, however, I then have 4 or 5 individual commits that weren't associated with a PR. Because I often find myself doing that.

Then the changelog script could categorize the pulls into their normal categories like they do now with the labels, but also list the individual commits.

Thanks for describing this further. I feel like it's such a super rare edge case. That said. prectically to achieve this only this place here would need to be updated:

• https://github.com/mikepenz/release-changelog-builder-action/blob/develop/src/pr-collector/prCollector.ts#L116-L129

Oh, one more thing and then I'm done, just as a future update (unless it's possible now). In commit mode, everything goes into its correct category, ie: Features, Bugs, etc.

Maybe some type of way to reverse the pattern so that it will categorize properly like it does now, but when it actually places that item in the list, it removes the beginning of the string. Because it looks sort of odd to have a list like this:

Features

• feat: added something
• feat: something else new
• feat: something else.

Whereas, stripping out the front word, the result would be:

Features

• added something
• something else new
• something else

So the front word would be removed, but go into the correct category.

And then in the workflow config

with:
  fromTag: "${{ env.TAG_LAST }}"
  toTag: ${{ github.ref }}
  configuration: ".github/changelog-configuration.json"
  stripCategory: true           <-----------------------

That's just something else thrown out there, but the rest works perfect. Makes my life a hell of a lot easier now.

There are actually multiple ways to achieve this already. So the "legacy" solution to achieve this would be via the transformers you can define, which are a regex transforming any "entry" using that regex.

However since v4 I actually prefer to use the placeholder feature. Which allows you to construct any placeholder from the source PR (or fake PR based on commit information).

A sample config for this (not exactly your target, but it kinda shows the way it works)

{
  "pr_template": "- ${{CLEAN_TITLE}}",
  "custom_placeholders": [
    {
      "name": "CLEAN_TITLE",
      "source": "TITLE",
      "transformer": {
        "pattern": "\\s*(\\[.+?(\\] ))?([\\S\\s]*)",
        "target": "$3"
      }
    }
  ]
}

With all of that. Not sure if you went that path, but I highly recommend to follow the Local Testing instructions, which allows you to iterate much much faster, and test the configs without the need to re-run the action in GitHub Actions:

• https://github.com/mikepenz/release-changelog-builder-action/tree/9ab35ce48da5a8bcebd04044fe09f4833985c872?tab=readme-ov-file#local-testing-

[Aetherinox]

Ah interesting, thanks. I'll try out the placeholders.

And yeah, I need a local test environment. Right now I have Act installed locally which runs via Docker, but I've used it very little, haven't had a lot of time to play with it. Thus far I've only used act to test out generating checksums from the distribution files.

Because working on this stuff live and having to keep pushing releases is eating up my time and it's just so damn slow.

I'll have to sit down tomorrow and play with placeholders and see what I can come up with, because if I can strip out the front category title from each commit message, then everything would be perfect.

I'd assume with placeholders, I'd just have to do the reverse of what I did with the pattern I pasted earlier. Having to filter out everything that isn't the front label, and then applying that to a new placeholder variable and using that in my template.

[mikepenz]

@Aetherinox

Used act before myself, it is fine for full workflows, but I figured for fine tuning release notes for various usecases it just didn't make sense, also to be honest. There is no reason this action has to be run in the context of a workflow. So running it for example in GitHub Codespaces (which is btw. how I develop it) allows you to get it bootstrapped super fast.

Start the codespace, create a token for your repo, and set-up a singular test case, and you are ready to roll.

And yes that's exactly what would be needed, you can either do the inverse to strip the front out. v5 will actually add (in my opinion) a simpler form of regex usage based on how regexr does it. but that's still alpha :D

[Aetherinox]

That was actually my issue with Act. It seemed very.... heavy, for what I was trying to do. I had to jump through a lot of stuff in docker, open up ports, and then it has this time out feature that if it cannot connect to something, it attempts to retry the connection 5 times, and there's a 30 second delay between each try, with no way to stop it, even CTRL Z doesn't work. So you either have to wait, or you have to kill terminal and re-launch.

Question, for

"target": "$3"

Is the value dependent on how many times the input has been chopped? So if I create a pattern to chop up the commit from the label such as:

fix: Did something

Would that mean that it brakes that up into

fix:     Did something
$1       $2

am I understanding that correctly?

[mikepenz]

the $3 refers to the group in the regex.

So for your case, I suppose you wanna match only the text after:

[Aetherinox]

Ah ok, so for mine, I'd use $2? Since group 2 is the text that I'm looking to return?

Then I guess one more question, then I'll go away, promise.

Currently, for the json template, I use

"template": "## What's New\n<sup>Version began on: #{{FROM_TAG_DATE}} - #{{TO_TAG_DATE}}</sup>\n<br />\n### File Changes\nThis release contains the folllowing edits:\n<ul><li><a href='#{{RELEASE_DIFF}}'>Changed files</a>  : <b>#{{CHANGED_FILES}}</b> </li><li>Commits : <b>#{{COMMITS}}</b> </li><li>Additions : <b>#{{ADDITIONS}}</b></li><li>Deletions : <b>#{{DELETIONS}}</b></li>\n<br />\n</ul>\n\n### Commits (#{{UNCATEGORIZED_COUNT}})\n#{{UNCATEGORIZED}}\n<br />\n\n### Pull Requests\n#{{CHANGELOG}}\n\n<br />\n\n",

All those commits are appearing in my release from UNCATEGORIZED, so how would I go about using a placeholder to filter out just the specific text of each commit? Because UNCATEGORIZED seems to just group all commits together.

Well, or CHANGELOG, depending on if I'm doing PR or commits.

Since all the commits appear based on that one variable.

[mikepenz]

So the placeholder you define will become a template variable you can use in your changelog.

{
  "pr_template": "- ${{CLEAN_TITLE}}",
  "custom_placeholders": [
    {
      "name": "CLEAN_TITLE",
      "source": "TITLE",
      "transformer": {
        "pattern": "\\s*(\\[.+?(\\] ))?([\\S\\s]*)",
        "target": "$3"
      }
    }
  ]
}

So in this example you define a new template placeholder called CLEAN_TITLE and you update the pr_template to use this for each entry (this also applies to commit based mode, given that every single commit becomes a "Fake" PR)

and those are then used for every single entry regardless of it being categorized or not categorized.

The fact that all of them are UNCATEGORIZED would mean none of your commits got categorized (e.g. no labels were available) is this really what you intend?

[Aetherinox]

Oh, no, maybe I completely screwed up explaining that.

So I understand that CLEAN_TITLE becomes a variable I can then use. But when my list of commits are generated right now, they all appear from a single variable in the template.

Every single commit comes from the single variable #{{UNCATEGORIZED}} or #{{CHANGELOG}}, depending on if that commit has a label.

So what I mean is, if every single commit shows up from that single variable, how would I use my new variable?

As a coder, I'd assume in order to manipulate #{{CHANGELOG}} or #{{UNCATEGORIZED}}, I'd have to loop through every entry and modify the title that way. But since that isn't a thing with workflows, how am I able to use CLEAN_TITLE, if every single commit shows up from one of those two variables above.

Sorry, maybe I'm explaining it horribly, it's really early. Or maybe I'm missing something.

Do I remove #{{CHANGELOG}} and not use that at all? And replace it with #{{CLEAN_TITLE}} in the example?

[mikepenz]

Oh I understand now.

Let me explain it differently.

So both: #{{UNCATEGORIZED}} and #{{CHANGELOG}} are constructed templates, hosting the entries as a result of the pr_template conversion.

The action differentiates between top level and PR level templates. PLACEHOLDERs are commonly PR level templates, which can be used in the pr_template. while CHANGELOG is a top level template.

So:

• TITLE: fix: test title, CLEAN_TITLE: test title in a pr_template:
  • #{{TITLE}} -> fix: test title
  • #{{CLEAN_TITLE}} -> test title
  • #{{TITLE}} // #{{CLEAN_TITLE}} -> fix: test title // test title

so no iteration from your side needed, just you defining the CLEAN_TITLE to be used in the pr_template which is then automatically used inside the CHANGELOG

[Aetherinox]

Ahhhhhh, ok, that explains it perfectly.

So in order to get the result I'm looking for, if I'm using CLEAN_TITLE for the example, then I'd do:

"pr_template": "- #{{CLEAN_TITLE}} : #{{MERGE_SHA}} @#{{AUTHOR}}",

Which would yield the result

test title : #000000 @Aetherinox

And those placeholders only having to do with pr_template

So I'd assume the following is correct:

    "custom_placeholders": [
        {
          "name": "TITLE_ONLY",
          "source": "TITLE",
          "transformer": {
            "pattern": "^(\\w+:\\s?)?(.+)",
            "target": "$2"
          }
        }
      ],

and then I'd use TITLE_ONLY as my value in pr_template.

[mikepenz]

Yes correct. However #{{MERGE_SHA}} will refer to the commit hash of commits in this case.

And yes correct.

[Aetherinox]

Yeah, the SHA hash is fine. I've been using that thus far, and it's nice because the long hash gets cut down in the changelog to roughly 6 or so digits.

I appreciate the long episode of help. You've made my life a lot easier, both your script and your help.

For years, I've been manually typing out my releases, and the ability to just click and have it all done for me is a huge upgrade.

So I really appreciate everything you've explained, greatly.

[mikepenz]

Glad I could help.

And really appreciate your kind words. It's why I created this action, allowing a tool with all the flexibility to cover most of the usecases out there which are not supported by the standard tools.

Also here's an upcoming PR: https://github.com/mikepenz/release-changelog-builder-action/pull/1312 which adds "HYBRID" mode, and this branch also has a test-case which will be able to convert conventional commits to cleaned titles. Worth noting that all those things will be in v5, as it has some minor breaking changes.

[Aetherinox]

Nice, I saw your released of 5 alpha, and I want to try it, just afraid it's going to cost me another 2-3 hours of development. It took a lot of patience last night to get things exactly how I want lol

and I've still got to try out the placeholders today to see if I did it properly.

I'm also debating on moving over from commitMode and doing PRs instead. Because I've realized in commit mode, it lists every single commit, even ones that would mean nothing to the end-user.

At least with PRs, I can commit as much as I want, and only the task will be listed.

Just a question, is their a specific type of regex this accepts? As an example: I tried to modify the regex

    "custom_placeholders": [
        {
          "name": "TITLE_ONLY",
          "source": "TITLE",
          "transformer": {
            "method": "regexr",
            "pattern": ": ?(.+)|(\\(.+)",
            "target": "$2"
          }
        }
      ],

which should do the following: https://regex101.com/r/WIeq2y/1

feat(something): text
feat: another word

would return

(something): text
another word

And that doesn't work.

However, this works

    "pr_template": "- ${{TITLE_ONLY}} : #{{MERGE_SHA}} @#{{AUTHOR}}",
    "empty_template": "- No major changes to address in this release",
    "custom_placeholders": [
        {
          "name": "TITLE_ONLY",
          "source": "TITLE",
          "transformer": {
            "method": "regexr",
            "pattern": "(\\w+(\\(.+\\))?: ?)?(.+)",
            "target": "$2 $3"
          }
        }
      ],

Both in regex101 do the same thing, but this action seems to only favor the second one.

[mikepenz]

I can understand. :D

I mean what you can do is that you filter out things which don't have a label. So only categorized things show up.

Given Actions run with Node20 it supports all regex patterns supported by node20. However there are a few caveats. Regexr actually does some special sauce to enable their List engine, which is non standard to javascript regex functions.

The action as of v4 supports 2 modes: replace(default), match. Which in your case the replace function is used. You can try out those core javascript functions here:

• https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/replace
• https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/match

Action v5 introduced the regexr method which you defined (however you will fallback to replace as v4 didn't have that one). regexr has a special function which extracts the individual groups and then "manually" replaces the target string. However I observed that most people use regexr so it made sense to offer a mode which works like they will see there. (while replace functions slightly differently.

You can see the core logic on that here:

• https://github.com/mikepenz/release-changelog-builder-action/blob/develop/src/pr-collector/regexUtils.ts#L54-L132

[mikepenz]

For v5 I added a testcase covering conventional commits: https://github.com/mikepenz/release-changelog-builder-action/blob/fb688cc2f5e3e548a1b67e9282bf273ad1f87e54/__tests__/releaseNotesBuilderPull.test.ts#L367-L408

Which I hope will be valuable :)

[Aetherinox]

Interesting. Thanks for the info.

I may just pop over to my test repo and migrate to 5 and play with it. Going to need to migrate to it eventually anyway, so I might as well work out the tweaks so that it'll be less of a pain later.

I found the regexr from one of your examples, so I assumed it worked on v4 (I think it was in the readme).

So in v5 for method, you mean the options available are replace, match, and regexr, if I understand correctly.

[mikepenz]

Yes I added it yesterday to the latest alpha of v5 (the README shows the latest dev state, if you want to only see the old state of the repo I'd advice to open the specific tag you use)

E.g v4.2.0: https://github.com/mikepenz/release-changelog-builder-action/tree/v4.2.0

Correct, v5 adds regexr as method using the behaviour of the online tool.

Ultimately also replace will work fine for you, you can also use the replace javascript documentation if you wanna test patterns quickly.

[mikepenz]

Decided to simplify the local testing instructions further on the latest develop branch: https://github.com/mikepenz/release-changelog-builder-action?tab=readme-ov-file#local-testing-

So now it's basically:

• Click the Open in Codespaces button
• Open the __tests__/demo/demo.test.ts file
• (For private repos) Provide a github token
• Modify to what you want to test (e.g. provide your config, provide your repo, your from/to tag)
• Run with npm test -- demo.test.ts and see the output

[Aetherinox]

Yeah I'm definitely going to use the local testing environment and play with v5. Testing on Github is just painfully slow.

Right now I'm dealing with the headache of figuring out how to read modified files from a pull request, and being able to read the raw text inside each file searching for a particular string, and it's proving to be a huge pain.

Granted, I'm about three weeks old with Github actions, so I haven't had a lot of experience, which is why it takes me a bit to figure these things out. It's a lot different going from C# and Lua over to Github actions. But thus far, I've loved how easy it makes life.

[mikepenz]

Read modified files from a PR, for which usecase? Static code analysis?

It becomes nicer the more tools you create :D

[Aetherinox]

Yeah, for the PR.

I've only been doing actions for about two-ish weeks. So to be blunt; I'm still dumber than a box of rocks.

Most of my years on Github, I've done everything manually. Wrote out my release notes, manually built my applications, and pretty much done everything the hard way.

A few weeks ago I decided to take a glance at github actions, and I realized that I had been ignoring a huge feature of Github that can make my life way easier. I was blown away when I found out I could feed the source files to my repo, and Github Actions could automatically build my NodeJS application, and automatically release it. That blew my mind away.

So now I'm stuck in this rabbit hole of making everything automated.

I was writing an automated script to scan my PR files for security vulnerabilities, deprecated practices, etc. And I was unaware that by simply using the correct action type, I was automatically already viewing the PR files. I thought I was starting out in my main branch and had to navigate over to the PR files.

So the whole time I was trying to figure out how to see my PR files, I could already do it lol

But that's just my lack of experience with Actions, but thus far I love them. I could sit here for weeks developing tools, but probably defeats the purpose of saving time, if I'm now spending all that time writing tools.

---

Last question, it just hit me.

In commit mode, it lists every commit you've done, even if it's stuff you may not even need users to know about.

Is their a possibility with the filtering system, where if I give a commit a certain prefix for the commit message, it can exclude listing that item at all?

Such as if my commit message is

internal: do something to whatever

Then when the changelog is generated, anything prefixed with internal would just completely be ignored from the changelog when it's generated during the action.

[mikepenz]

I am sure you will figure all out :). As you learn all of these, don't forget to also consider security risks involved with adding actions and being selective on what you add.

(Personally I do an audit of every 3rd party action I use, specifically if they are not from Github. (Even for verified companies) - given actions have quite some visibility)

---

To your last question.

Yes absolutely. So you can do multiple things there:

• Either you only have categories with labels, and you don't have an uncategorized one (so they would just not show up anywhere)
• or you define a global ignore_label (see the configuration definition for it)
• you can also define exclude_labels for every category

Most of the time I go with either only having specific categories (and not showing anything not categorized) or I go with the ignore label:

[Aetherinox]

Ooooooh, so for not displaying Uncategorized, I assume I'd just delete the variable `#{{UNCATEGORIZED}}``, and then it wouldn't show the ones that don't have a recognized cat?

I'm going to also play with ignore_labels and exclude labels that way I have a full understanding of what options I have.

I'm trying to migrate over to the pull request practice, but it seems like a lot to manage the pull requests plus the issues, because I like to use issues to categorize what I have to work on, and then I assign those labels, and place them into my "Project" in the trello-like cards so I can see exactly what I've got going on.

[mikepenz]

Exactly :) UNCATEGORIZED is the container for all the things which didn't end up in a category.

[Aetherinox]

Hey, one more thing I could bother you. Appreciate all the help.

Updating my templates, and I noticed that TO_TAG_DATE never returns anything and is blank.

I have it utilized as:

"## What's New\n<sup>Version began on: #{{FROM_TAG_DATE}} - #{{TO_TAG_DATE}}</sup>\n<br />

and I've ensured I satisfied the requirements per the readme:

- name: Build Changelog
  id: changelog
  uses: mikepenz/release-changelog-builder-action@v4
  with:
    fromTag: "${{ env.TAG_LAST }}"
    toTag: ${{ github.ref }}
    configuration: ".github/changelog-configuration.json"
    ignorePreReleases: false
    commitMode: ${{ inputs.CHANGELOG_MODE_COMMIT }}
    fetchReleaseInformation: true

Which has fetchReleaseInformation enabled.

[mikepenz]

@Aetherinox please open a new issue for that one :)

Also I think we can close this thread.

[Aetherinox]

Appreciate it! Just didn't want to bug ya.