conventional-changelog-conventionalcommits v8.0.0 breaks semantic release

ryancausey commented 2 months ago

It looks like v8.0.0 of conventional-changelog-conventionalcommits was released about an hour ago. This causes the semantic-release "generateNotes" step of plugin "@semantic-release/release-notes-generator" to fail:

$ npm --version
10.5.0
$ semantic-release --version
23.0.8
$ semantic-release
[11:48:10 PM] [semantic-release] › ℹ  Running semantic-release version 23.0.8
[11:48:11 PM] [semantic-release] › ✔  Loaded plugin "verifyConditions" from "@semantic-release/changelog"
[11:48:11 PM] [semantic-release] › ✔  Loaded plugin "verifyConditions" from "@semantic-release/exec"
[11:48:11 PM] [semantic-release] › ✔  Loaded plugin "verifyConditions" from "@semantic-release/git"
[11:48:11 PM] [semantic-release] › ✔  Loaded plugin "verifyConditions" from "@semantic-release/gitlab"
[11:48:11 PM] [semantic-release] › ✔  Loaded plugin "analyzeCommits" from "@semantic-release/commit-analyzer"
[11:48:11 PM] [semantic-release] › ✔  Loaded plugin "analyzeCommits" from "@semantic-release/exec"
[11:48:11 PM] [semantic-release] › ✔  Loaded plugin "verifyRelease" from "@semantic-release/exec"
[11:48:11 PM] [semantic-release] › ✔  Loaded plugin "generateNotes" from "@semantic-release/release-notes-generator"
[11:48:11 PM] [semantic-release] › ✔  Loaded plugin "generateNotes" from "@semantic-release/exec"
[11:48:11 PM] [semantic-release] › ✔  Loaded plugin "prepare" from "@semantic-release/changelog"
[11:48:11 PM] [semantic-release] › ✔  Loaded plugin "prepare" from "@semantic-release/exec"
[11:48:11 PM] [semantic-release] › ✔  Loaded plugin "prepare" from "@semantic-release/git"
[11:48:11 PM] [semantic-release] › ✔  Loaded plugin "publish" from "@semantic-release/exec"
[11:48:11 PM] [semantic-release] › ✔  Loaded plugin "publish" from "@semantic-release/gitlab"
[11:48:11 PM] [semantic-release] › ✔  Loaded plugin "addChannel" from "@semantic-release/exec"
[11:48:11 PM] [semantic-release] › ✔  Loaded plugin "success" from "@semantic-release/exec"
[11:48:11 PM] [semantic-release] › ✔  Loaded plugin "success" from "@semantic-release/gitlab"
[11:48:11 PM] [semantic-release] › ✔  Loaded plugin "fail" from "@semantic-release/exec"
[11:48:11 PM] [semantic-release] › ✔  Loaded plugin "fail" from "@semantic-release/gitlab"
[11:48:15 PM] [semantic-release] › ✔  Run automated release from branch master on repository [https://gitlab-ci-token:[secure]@gitlab.com/foo/bar.git](https://gitlab-ci-token:%5Bsecure%5D@gitlab.com/foo/bar.git)
[11:48:15 PM] [semantic-release] › ✔  Allowed to push to the Git repository
[11:48:15 PM] [semantic-release] › ℹ  Start step "verifyConditions" of plugin "@semantic-release/changelog"
[11:48:15 PM] [semantic-release] › ✔  Completed step "verifyConditions" of plugin "@semantic-release/changelog"
[11:48:15 PM] [semantic-release] › ℹ  Start step "verifyConditions" of plugin "@semantic-release/exec"
[11:48:15 PM] [semantic-release] › ✔  Completed step "verifyConditions" of plugin "@semantic-release/exec"
[11:48:15 PM] [semantic-release] › ℹ  Start step "verifyConditions" of plugin "@semantic-release/git"
[11:48:15 PM] [semantic-release] › ✔  Completed step "verifyConditions" of plugin "@semantic-release/git"
[11:48:15 PM] [semantic-release] › ℹ  Start step "verifyConditions" of plugin "@semantic-release/gitlab"
[11:48:15 PM] [semantic-release] [@semantic-release/gitlab] › ℹ  Verify GitLab authentication (https://gitlab.com/api/v4)
[11:48:15 PM] [semantic-release] › ✔  Completed step "verifyConditions" of plugin "@semantic-release/gitlab"
[11:48:15 PM] [semantic-release] › ℹ  Found git tag v1.4.0 associated with version 1.4.0 on branch master
[11:48:15 PM] [semantic-release] › ℹ  Found 1 commits since last release
[11:48:15 PM] [semantic-release] › ℹ  Start step "analyzeCommits" of plugin "@semantic-release/commit-analyzer"
[11:48:15 PM] [semantic-release] [@semantic-release/commit-analyzer] › ℹ  Analyzing commit: fix: make the esg rule name not clash with other domains
This was not applying properly because the rule has the same name as a
rule created by a different AWS account on the same bus.
[11:48:15 PM] [semantic-release] [@semantic-release/commit-analyzer] › ℹ  The release type for the commit is patch
[11:48:15 PM] [semantic-release] [@semantic-release/commit-analyzer] › ℹ  Analysis of 1 commits complete: patch release
[11:48:15 PM] [semantic-release] › ✔  Completed step "analyzeCommits" of plugin "@semantic-release/commit-analyzer"
[11:48:15 PM] [semantic-release] › ℹ  Start step "analyzeCommits" of plugin "@semantic-release/exec"
[11:48:15 PM] [semantic-release] › ✔  Completed step "analyzeCommits" of plugin "@semantic-release/exec"
[11:48:15 PM] [semantic-release] › ℹ  The next release version is 1.4.1
[11:48:15 PM] [semantic-release] › ℹ  Start step "verifyRelease" of plugin "@semantic-release/exec"
[11:48:15 PM] [semantic-release] › ✔  Completed step "verifyRelease" of plugin "@semantic-release/exec"
[11:48:15 PM] [semantic-release] › ℹ  Start step "generateNotes" of plugin "@semantic-release/release-notes-generator"
[11:48:15 PM] [semantic-release] › ✘  Failed step "generateNotes" of plugin "@semantic-release/release-notes-generator"
[11:48:15 PM] [semantic-release] › ✘  An error occurred while running semantic-release: RangeError: Invalid time value
    at committerDate (/usr/local/lib/node_modules/semantic-release/node_modules/conventional-changelog-writer/index.js:80:30)
    at /usr/local/lib/node_modules/semantic-release/node_modules/conventional-changelog-writer/lib/util.js:202:17
    at Array.forEach (<anonymous>)
    at processCommit (/usr/local/lib/node_modules/semantic-release/node_modules/conventional-changelog-writer/lib/util.js:198:31)
    at /usr/local/lib/node_modules/semantic-release/node_modules/conventional-changelog-writer/index.js:123:32 {
  pluginName: '@semantic-release/release-notes-generator'
}
RangeError: Invalid time value
    at committerDate (/usr/local/lib/node_modules/semantic-release/node_modules/conventional-changelog-writer/index.js:80:30)
    at /usr/local/lib/node_modules/semantic-release/node_modules/conventional-changelog-writer/lib/util.js:202:17
    at Array.forEach (<anonymous>)
    at processCommit (/usr/local/lib/node_modules/semantic-release/node_modules/conventional-changelog-writer/lib/util.js:198:31)
    at /usr/local/lib/node_modules/semantic-release/node_modules/conventional-changelog-writer/index.js:123:32 {
  pluginName: '@semantic-release/release-notes-generator'
}

Reverting conventional-changelog-conventionalcommits back to the v7.x.x series makes this release run fine again.

jedwards1211 commented 2 months ago

I had the error even with conventional-changelog-conventionalcommits v7.x.x; it seems like it was being caused because the preset I was using with @semantic-release/commit-analyzer didn't match the preset I was using with @semantic-release/release-notes-generator. When I made them match, the error went away with v8.0.0.

ryancausey commented 2 months ago

I had the error even with conventional-changelog-conventionalcommits v7.x.x; it seems like it was being caused because the preset I was using with @semantic-release/commit-analyzer didn't match the preset I was using with @semantic-release/release-notes-generator. When I made them match, the error went away with v8.0.0.

I'm not sure what that means. I simply have a preset: conventionalcommits at the top of my .releaserc.yml file, so I would assume they all share the same preset?

jedwards1211 commented 2 months ago

@ryancausey Oh, I'm not sure, I'm configuring all the plugins explicitly, and the plugins accept the preset option.

travi commented 2 months ago

the conventional-changelog packages released new major versions. that means they include breaking changes. since they were just released, no work has been done to make adjustments to account for the changes in semantic-release.

if someone would be willing to investigate what changes would be needed, that could help us adjust for these changes sooner

jedwards1211 commented 2 months ago

@travi yeah that makes sense. If we need to customize the preset though, we have to install it, and then it's easy to install the wrong version... so should we make this package check the version of the preset at runtime, and at least print a warning? I would almost suggest we put the compatible version in optionalDependencies, but package managers, confoundingly, install optional dependencies by default. So the best I can think to do is a runtime check. I'd be happy to make a PR.

travi commented 1 month ago

@travi yeah that makes sense. If we need to customize the preset though, we have to install it, and then it's easy to install the wrong version... so should we make this package check the version of the preset at runtime, and at least print a warning? I would almost suggest we put the compatible version in optionalDependencies, but package managers, confoundingly, install optional dependencies by default. So the best I can think to do is a runtime check. I'd be happy to make a PR.

I understand the desire, but npm doesn't provide a great way to define these sorts of relationships. I'm open to proposals as a separate issue if there are options that I'm not aware of, but this is a difficult one to solve. I think the closest might be peerDependencies with meta defining the optional part, but I think there are complexities that make that not ideal too

travi commented 1 month ago

For folks landing here for various breakages related to updating conventional-commits packages before semantic-release is updated to handle the breaking changes, I'm trying to consolidate the conversation in this thread.

Until I get more organized about that, please see my suggestions in https://github.com/semantic-release/semantic-release/issues/3292#issuecomment-2099434430

jedwards1211 commented 1 month ago

@travi not sure you saw my suggestion to check and warn at runtime if the preset package being used is not a known compatible version, would you be okay with a PR to do that?

travi commented 1 month ago

not sure you saw my suggestion to check and warn at runtime if the preset package being used is not a known compatible version, would you be okay with a PR to do that?

sorry. i did, but forgot to comment about that approach. it doesnt seem to me like there is a clean approach there either that is maintainable and still accomplishes the goal for new breaking changes. our approach to true peer dependencies and node versions is to keep them open ended, so we would have to release a new version with updated ranges once we are aware that a new major version is incompatible, which also requires responding fairly quickly to be most helpful. each of the conventional-commits packages has its own individual major version (an approach that i agree with), so we would have to handle each one individually. this also means more investigation when trying to release a version for such incompatibility communication.

in the end, i think this almost entirely comes down to folks installing latest versions in pipelines without pinning a major version. would be open to suggestions to make this suggestion more clear and discoverable in docs or elsewhere in a general way that doesnt require maintainer action each time that breaking changes are released in conventional-commits packages.

jedwards1211 commented 1 month ago

think this almost entirely comes down to folks installing latest versions in pipelines without pinning a major version.

In my case it wasn't that - I was installing conventional-changelog-conventionalcommits for the first time so that I could try to customize the behavior for a monorepo.

I can understand where you're coming from though. I can also understand the way all these different packages are structured. But - figuring out how to even customize everything and follow the control flow of release notes generation has left me thinking the work of fetching and parsing git commits and generating a changelog from them is spread out across too many packages for its own good. Maybe I would feel differently if these were all TS projects or the configuration was done via a strongly typed TS API rather than an unvalidated JSON DSL. It's pretty hairy to dig into how the configuration is getting passed around and used. And at least half of the code is for implementing that DSL that isn't really capable of doing what I want anyway, I wish I could just provide a function for filtering and categorizing commits by type and scope, etc, which would be way more flexible and require way less library code.

DenisBalan commented 1 month ago

We have the same issue.. Decided to downgrade for only package which is causing troubles to conventional-changelog-conventionalcommits@7.0.2

...
npm install semantic-release \
@semantic-release/changelog \
@semantic-release/git \
conventional-changelog-conventionalcommits@7.0.2
...

anthony-nhs commented 1 month ago

in the end, i think this almost entirely comes down to folks installing latest versions in pipelines without pinning a major version. would be open to suggestions to make this suggestion more clear and discoverable in docs or elsewhere in a general way that doesnt require maintainer action each time that breaking changes are released in conventional-commits packages.

I agree that is what people should do. However, the problem I found is that is currently hard to test if a major version bump will cause something to fail as we have semantic-release only running on our main branch. Implementing https://github.com/semantic-release/semantic-release/issues/3278 would allow us to at least run with dry-run in a branch where we upgrade versions and see if anything breaks

travi commented 1 month ago

https://github.com/semantic-release/semantic-release/releases/tag/v24.0.0-beta.2 should resolve this problem and enable use of the latest conventional-changelog packages.

it would be appreciated if folks following this issue that have experienced problems with the new major versions of the conventional-changelog plugins and let us know if this version resolves the problem for you

ashvardanian commented 1 month ago

Hi @travi! I've just tried half of the possible configuration arguments and severely polluted the main branch of one of the affected packages, but still can't make Semantic Release to work... The last time it worked was in mid April. Any recommendations?

travi commented 1 month ago

@ashvardanian the new beta of semantic-release is intended to work with the latest majors of conventional-changelog presets. As mentioned in the release notes, that means older version will no longer work. For the eslint preset, it looks like v6.0.0 is the latest version, but you're project is still configured to use v3.

alexey-sh commented 1 month ago

So what is the plan?

strausmann commented 1 month ago

I just asked myself that. currently my Semantic release is no longer running. my gitlab ci pipelines are standing still.

travi commented 1 month ago

There are two options available

Pin the major versions of the packages you are installing alongside semantic-release until we release a new version that supports the latest conventional-changelog packages
Try out the beta that should support those new versions and let us know if it works in your context. That will help give us the confidence to promote the fix for everyone to use (even after we promote the fix, you should still pin the major version of those extra packages to avoid this situation next time)

ashvardanian commented 1 month ago

Hi @travi! Thanks for suggestions!

I've bumped all the versions in my package.json, which now starts like this:


{
    "name": "simsimd-ci",
    "version": "1.0.0",
    "devDependencies": {
        "@semantic-release/exec": "^6.0.3",
        "@semantic-release/git": "^10.0.1",
        "@semantic-release/commit-analyzer": "^12.0.0",
        "@semantic-release/release-notes-generator": "^13.0.0",
        "@semantic-release/github": "^10.0.5",
        "conventional-changelog-eslint": "^6.0.0",
        "semantic-release": "^24.0.0-beta.2"
    },
    "release": {
        "debug": true,
        "ci": false,
        "dryRun": false,
        "private": true,
        "branches": [
            "main"
        ],
        "plugins": [

Then, when I run the following command locally:

npx --prefix .github/workflows semantic-release  --no-ci --no-npm --debug

I'm still getting errors like:

[7:51:24 PM] [semantic-release] › ✔  Completed step "verifyConditions" of plugin "@semantic-release/npm"
[7:51:24 PM] [semantic-release] › ℹ  Start step "verifyConditions" of plugin "@semantic-release/github"
[7:51:24 PM] [semantic-release] [@semantic-release/github] › ℹ  Verify GitHub authentication
[7:51:24 PM] [semantic-release] › ✘  Failed step "verifyConditions" of plugin "@semantic-release/github"
[7:51:24 PM] [semantic-release] › ℹ  Start step "fail" of plugin "@semantic-release/github"
[7:51:24 PM] [semantic-release] [@semantic-release/github] › ℹ  Verify GitHub authentication
[7:51:24 PM] [semantic-release] › ✘  Failed step "fail" of plugin "@semantic-release/github"
[7:51:24 PM] [semantic-release] › ✘  ENOGHTOKEN No GitHub token specified.
A GitHub personal token must be created and set in the GH_TOKEN or GITHUB_TOKEN environment variable on your CI environment.

Please make sure to create a GitHub personal token and to set it in the GH_TOKEN or GITHUB_TOKEN environment variable on your CI environment. The token must allow to push to the repository ashvardanian/SimSIMD.

[7:51:25 PM] [semantic-release] › ✘  ENOGHTOKEN No GitHub token specified.
A GitHub personal token must be created and set in the GH_TOKEN or GITHUB_TOKEN environment variable on your CI environment.

Same happens if I remove any GitHub-related dependencies or steps. How can I "dry-run" the semantic release pipeline locally to avoid polluting the release history and the main branch of my projects?

travi commented 1 month ago

You do not need to/should not install the commit-analyzer or release-notes-generator plugins as direct dependencies, since they are already dependencies of semantic-release itself. If you test with npm ls <package-name>, you'll see that has resulted in you having multiple versions installed. Removing those direct dependencies should resolve your problem

ashvardanian commented 1 month ago

@travi, I've just tried that locally and still face same issues.

$ npm cache clean --force
$ rm -rf .github/workflows/node_modules .github/workflows/package-lock.json
$ nm install --ignore-scripts --legacy-peer-deps --prefix .github/workflows
$ npx --prefix .github/workflows semantic-release  --no-ci --no-npm --debug
...
[1:03:58 AM] [semantic-release] › ✔  Completed step "verifyConditions" of plugin "@semantic-release/npm"
[1:03:58 AM] [semantic-release] › ℹ  Start step "verifyConditions" of plugin "@semantic-release/github"
[1:03:58 AM] [semantic-release] [@semantic-release/github] › ℹ  Verify GitHub authentication
[1:03:58 AM] [semantic-release] › ✘  Failed step "verifyConditions" of plugin "@semantic-release/github"
[1:03:58 AM] [semantic-release] › ℹ  Start step "fail" of plugin "@semantic-release/github"
[1:03:58 AM] [semantic-release] [@semantic-release/github] › ℹ  Verify GitHub authentication
[1:03:58 AM] [semantic-release] › ✘  Failed step "fail" of plugin "@semantic-release/github"
[1:03:58 AM] [semantic-release] › ✘  ENOGHTOKEN No GitHub token specified.
A GitHub personal token must be created and set in the GH_TOKEN or GITHUB_TOKEN environment variable on your CI environment.

What else would you recommend trying?

ryancausey commented 1 month ago

https://github.com/semantic-release/semantic-release/releases/tag/v24.0.0-beta.2 should resolve this problem and enable use of the latest conventional-changelog packages.

it would be appreciated if folks following this issue that have experienced problems with the new major versions of the conventional-changelog plugins and let us know if this version resolves the problem for you

I've tested with this and it resolved my issue.

HenrikPoulsen commented 1 month ago

Would be nice if this was something that wasn't always discovered aftter an update was being made in a repo. Some sort of sanity check somewhere in the pipeline would be nice. I had a similar issue, but in my case it didn't get as far as the release-notes-generator and ended up not working in the commit-analyzer, which just didn't detect any changes and I spent a day trying to figure out what I had done wrong when tryring to setup conventionalcommits on the repo. And this is a setup I was planning to setup in a 100+ repos across many teams. It would be extremely costly if Dependabot or Renovate or whatever suggests they update to a new version of one of the plugins, and then several weeks after they've merged they notice that it's actually not working anymore because there's no errors etc. Pinning won't really solve the type of issue I am proposing there. Because you will have to update eventually. How are you supposed to do that safely? Preferably in an automated way. Like having a semantic-releasee verify that checks all the plugins and configs and complains if things look weird.

LeleDallas commented 1 month ago

Would be nice if this was something that wasn't always discovered aftter an update was being made in a repo. Some sort of sanity check somewhere in the pipeline would be nice. I had a similar issue, but in my case it didn't get as far as the release-notes-generator and ended up not working in the commit-analyzer, which just didn't detect any changes and I spent a day trying to figure out what I had done wrong when tryring to setup conventionalcommits on the repo. And this is a setup I was planning to setup in a 100+ repos across many teams. It would be extremely costly if Dependabot or Renovate or whatever suggests they update to a new version of one of the plugins, and then several weeks after they've merged they notice that it's actually not working anymore because there's no errors etc. Pinning won't really solve the type of issue I am proposing there. Because you will have to update eventually. How are you supposed to do that safely? Preferably in an automated way. Like having a semantic-releasee verify that checks all the plugins and configs and complains if things look weird.

A command like semantic-releasee verify will be a great feature!

BinToss commented 1 month ago

@travi, I've just tried that locally and still face same issues.

$ npm cache clean --force
$ rm -rf .github/workflows/node_modules .github/workflows/package-lock.json
$ nm install --ignore-scripts --legacy-peer-deps --prefix .github/workflows
$ npx --prefix .github/workflows semantic-release  --no-ci --no-npm --debug
...
[1:03:58 AM] [semantic-release] › ✔  Completed step "verifyConditions" of plugin "@semantic-release/npm"
[1:03:58 AM] [semantic-release] › ℹ  Start step "verifyConditions" of plugin "@semantic-release/github"
[1:03:58 AM] [semantic-release] [@semantic-release/github] › ℹ  Verify GitHub authentication
[1:03:58 AM] [semantic-release] › ✘  Failed step "verifyConditions" of plugin "@semantic-release/github"
[1:03:58 AM] [semantic-release] › ℹ  Start step "fail" of plugin "@semantic-release/github"
[1:03:58 AM] [semantic-release] [@semantic-release/github] › ℹ  Verify GitHub authentication
[1:03:58 AM] [semantic-release] › ✘  Failed step "fail" of plugin "@semantic-release/github"
[1:03:58 AM] [semantic-release] › ✘  ENOGHTOKEN No GitHub token specified.
A GitHub personal token must be created and set in the GH_TOKEN or GITHUB_TOKEN environment variable on your CI environment.

What else would you recommend trying?

A. Create and securely store a GitHub Personal Access Token that has repo scope (classic token) or "Contents" repository permissions (write) (fine-grained). Then, try npx --prefix .github/workflows cross-env GITHUB_TOKEN=**** semantic-release --no-ci --no-npm --debug B. Remove the github plugin from your main config. C. Create a separate config file which imports the main one and then options.plugins = options.plugins.filter(pluginSpec =>typeof pluginSpec === 'string' ? pluginSpec !== '@semantic-release/github' : pluginSpec[0] !== '@semantic-release/github'); before export default options. You may need to suppress a TypeScript 'readonly' error. Then, add --extends './SRNoGithub.js' to your CLI command.

imliuruiqi commented 1 month ago

FYI, tried on self-hosted gitlab with python project and it works as expected. 🎉

script:
    - npm install @semantic-release/gitlab @semantic-release/exec @semantic-release/changelog conventional-changelog-conventionalcommits @semantic-release/git -D
    - npx semantic-release@24.0.0-beta.2

configuration of semantic-release:

plugins:
  - "@semantic-release/commit-analyzer"
  - "@semantic-release/release-notes-generator"
  - - "@semantic-release/changelog"
    - changelogFile: CHANGELOG.md
  - - "@semantic-release/git"
    - assets:
        - CHANGELOG.md
  - - "@semantic-release/exec"
    - prepareCmd: "poetry version ${nextRelease.version} && poetry build"
  - - "@semantic-release/gitlab"
    - assets:
      - path: "dist/*.whl"
        type: "package"
      - path: "dist/*.gz"
        type: "package"
      - path: "CHANGELOG.md"
        type: "runbook"

preset: "conventionalcommits"

github-actions[bot] commented 1 month ago

:tada: This issue has been resolved in version 14.0.0 :tada:

The release is available on:

Your semantic-release bot :package::rocket:

travi commented 1 month ago

Thank you @sherbakovdev for contributing to getting this out the door and to @ryancausey and @imliuruiqi for testing out the fix and confirming that it solved the problem in your projects 🎉

semantic-release / release-notes-generator

conventional-changelog-conventionalcommits v8.0.0 breaks semantic release #633