search

jorisroovers / gitlint

Linting for your git commit messages
http://jorisroovers.github.io/gitlint
MIT License
806 stars 99 forks source link

0.19.0 Release Plan #367

Closed jorisroovers closed 1 year ago

jorisroovers commented 1 year ago

This issue will be used to track any overarching discussion and work related to 0.19.0.

Scope

As always, everything discussed in this issue is subject to sudden change at my sole discretion. No promises or hard commitments :-)

All related issues will be tagged with the 0.19.0 Milestone.

We also maintain a Kanban board on Github Projects, items marked as TODO are things I'd like to work on for the next release (but again, subject to change).

For 0.19.0, I want to focus on improving (modernizing) our build and CI/CD workflows, with the end-goal of reducing maintenance overhead and making it easier to release.

Timeline

As always, no specific timeline, but probably in the new year (2023).

webknjaz commented 1 year ago

@jorisroovers re:GHA — feel free to borrow some ideas from https://github.com/ansible/pylibssh/blob/devel/.github/workflows/build-test-n-publish.yml / https://github.com/cherrypy/cheroot/blob/main/.github/workflows/ci-cd.yml / https://github.com/sphinx-contrib/sphinxcontrib-towncrier/blob/master/.github/workflows/tox-tests.yaml. They implement some of the things I mentioned in https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/ and the action I've made for that, like publishing every commit to the default branch to TestPyPI to ensure the pipeline is healthy. Note that these are quite complex because I haven't yet migrated them to rely on reusable workflows but there's a lot of interesting ideas.

webknjaz commented 1 year ago

@jorisroovers as the first step, I've got you this PR https://github.com/jorisroovers/gitlint/pull/375 — it'll allow you to enable branch projection for one check job which ensures that all of the other jobs are green and there's no need to list each matrix factor in the repo settings.

jorisroovers commented 1 year ago

@webknjaz just a quick acknowledgement of all the issues you've commented on today. Really appreciate all the input!

Heads up that it will take me a while to get to all of them - my available time is limited - but will work through them all :-)

webknjaz commented 1 year ago

Yep, I noticed the disclaimers :) I was about to file a UX issue but ended up checking whether anybody suggested the same and seeing a few places where I might leave pointers or send a PR. FYI I'm quite proficient with GHA so feel free to tag me if you're stuck with something.

jorisroovers commented 1 year ago

Just released 0.19.0 🎉 https://github.com/jorisroovers/gitlint/releases/tag/v0.19.0

@harens can you publish for macos? Thanks!

harens commented 1 year ago

@jorisroovers thanks for the update. Just a heads up, this isn't a trivial update due to hatch requiring a git repo to build from (i.e. not just a standard tar/zip). This isn't normally too big of an issue, but it's made a bit more complicated since it's the gitlint-core directory that needs to be built, not the higher level gitlint directory.

Same issue appears on the Homebrew side: https://github.com/Homebrew/homebrew-core/pull/125001. I should be able to fix this for MacPorts, but I'll need some time to think about it.

harens commented 1 year ago

Should be all fixed for MacPorts (https://github.com/macports/macports-ports/commit/00073ee3d745618be5865b5d5781a33d8cd92aa9). Thanks again for the update @jorisroovers! 🎉

sigmavirus24 commented 1 year ago

Wait, you can't build a wheel from the sdist when using hatch VCS? That's super awful

harens commented 1 year ago

Wait, you can't build a wheel from the sdist when using hatch VCS? That's super awful

Example logs below on my machine. Yeah it's not ideal...

Building from PyPi sdist:

> cd gitlint_core-0.19.0
> hatch build
[sdist]
fatal: not a git repository (or any of the parent directories): .git
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/__main__.py", line 6, in <module>
    sys.exit(hatchling())
             ^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/cli/__init__.py", line 26, in hatchling
    command(**kwargs)
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/cli/build/__init__.py", line 75, in build_impl
    for artifact in builder.build(
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/builders/plugin/interface.py", line 93, in build
    self.metadata.validate_fields()
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 243, in validate_fields
    _ = self.version
        ^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 128, in version
    self._version = self._get_version()
                    ^^^^^^^^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 222, in _get_version
    core_metadata = self.core
                    ^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 174, in core
    metadata_hook.update(self.core_raw_metadata)
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/metadata_hook.py", line 45, in update
    urls[key] = formatter.format(url)
                ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/string.py", line 190, in format
    return self.vformat(format_string, args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/utils/context.py", line 127, in vformat
    result, _ = self._vformat(format_string, args, kwargs, used_args, 10)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/string.py", line 247, in _vformat
    result.append(self.format_field(obj, format_spec))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatchling/utils/context.py", line 145, in format_field
    return self.__formatters[formatter](value, data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/metadata_hook.py", line 39, in <lambda>
    'commit_hash': lambda *args: vcs_utils.get_commit_hash(self.root),
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Cnj9QvS_/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/vcs_utils.py", line 10, in get_commit_hash
    return subprocess.check_output(['git', 'rev-parse', 'HEAD'], cwd=root).decode('utf-8').strip()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'rev-parse', 'HEAD']' returned non-zero exit status 128.

Building from github tarball:

> cd gitlint-0.19.0/gitlint-core
> hatch build
[sdist]
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/__main__.py", line 6, in <module>
    sys.exit(hatchling())
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/cli/__init__.py", line 26, in hatchling
    command(**kwargs)
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/cli/build/__init__.py", line 81, in build_impl
    clean_only=clean_only,
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/builders/plugin/interface.py", line 93, in build
    self.metadata.validate_fields()
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/metadata/core.py", line 243, in validate_fields
    _ = self.version
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/metadata/core.py", line 128, in version
    self._version = self._get_version()
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/metadata/core.py", line 222, in _get_version
    core_metadata = self.core
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/metadata/core.py", line 170, in core
    self._version = self._get_version(metadata)
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/metadata/core.py", line 226, in _get_version
    version = self.hatch.version.cached
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/77YI8rZN/gitlint-core-build/lib/python3.7/site-packages/hatchling/metadata/core.py", line 1412, in cached
    raise type(e)(message) from None
LookupError: Error getting the version from source `vcs`: setuptools-scm was unable to detect version for /Users/harensamarasinghe/Downloads/gitlint-0.19.0.

Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.

For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=proj
jorisroovers commented 1 year ago

Hmm - I clearly didn't consider this case of rebuilding the wheel directly from the sdist. Although I vaguely recall trying this, perhaps after adopting hatch but before adopting hatch-vcs.

hatch-vcs does indeed need the git repo, and not just a shallow clone, but a full one (or at least a including a single tag, not sure): https://github.com/jorisroovers/gitlint/blob/014bfaeac8c75bf0a06b55e533a39a79b3c69cd3/.github/workflows/ci.yml#L28

I wonder whether this is hatch-vcs specific, or dictated by setuptools-scm - I'd have to dig in.

Perhaps @ofek or @RonnyPfannschmidt have thoughts here?

ofek commented 1 year ago

The latter, but one solution is to write it to a file

jorisroovers commented 1 year ago

@ofek can you provide a bit more detail or a pointer, so I can have a look? Thanks!

RonnyPfannschmidt commented 1 year ago

At least a single tag is needed

Git shallow clones are pretty much sabotage for anything working with Metadata

If hatch sdists don't include a pkginfo it would need a fix

ofek commented 1 year ago

Actually, are you sure that source distributions aren't working for you correctly? It should work out-of-the-box

jorisroovers commented 1 year ago

Actually, are you sure that source distributions aren't working for you correctly? It should work out-of-the-box

I'll try it out myself tomorrow morning my time and do it bit more digging. Entirely possible I made a mistake :-)

ofek commented 1 year ago

@harens If you look at the beginning of your output it's trying to build a source distribution because without any arguments the default behavior is to build both that and a wheel, similar to the tool build. Instead, do hatch build -t wheel or python -m build --wheel

ofek commented 1 year ago

You would likely want to revert that MacPorts change

ofek commented 1 year ago

Actually there is a much deeper problem here: the source distribution contains nothing https://github.com/jorisroovers/gitlint/blob/014bfaeac8c75bf0a06b55e533a39a79b3c69cd3/pyproject.toml#L40-L41

sdist

harens commented 1 year ago

Instead, do hatch build -t wheel or python -m build --wheel

> wget https://files.pythonhosted.org/packages/29/59/c317a4a39657a4a76cd7dc0c9db4147cca57f13ebe11a469d8c6c0f88715/gitlint_core-0.19.0.tar.gz
> tar -xzvf gitlint_core-0.19.0.tar.gz
> hatch build -t wheel
fatal: not a git repository (or any of the parent directories): .git
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/__main__.py", line 6, in <module>
    sys.exit(hatchling())
             ^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/cli/__init__.py", line 26, in hatchling
    command(**kwargs)
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/cli/build/__init__.py", line 75, in build_impl
    for artifact in builder.build(
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/builders/plugin/interface.py", line 93, in build
    self.metadata.validate_fields()
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 243, in validate_fields
    _ = self.version
        ^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 128, in version
    self._version = self._get_version()
                    ^^^^^^^^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 222, in _get_version
    core_metadata = self.core
                    ^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 174, in core
    metadata_hook.update(self.core_raw_metadata)
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/metadata_hook.py", line 45, in update
    urls[key] = formatter.format(url)
                ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/string.py", line 190, in format
    return self.vformat(format_string, args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/utils/context.py", line 127, in vformat
    result, _ = self._vformat(format_string, args, kwargs, used_args, 10)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/string.py", line 247, in _vformat
    result.append(self.format_field(obj, format_spec))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatchling/utils/context.py", line 145, in format_field
    return self.__formatters[formatter](value, data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/metadata_hook.py", line 39, in <lambda>
    'commit_hash': lambda *args: vcs_utils.get_commit_hash(self.root),
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/harensamarasinghe/Library/Application Support/hatch/env/virtual/gitlint-core/Yar0NeOR/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/vcs_utils.py", line 10, in get_commit_hash
    return subprocess.check_output(['git', 'rev-parse', 'HEAD'], cwd=root).decode('utf-8').strip()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'rev-parse', 'HEAD']' returned non-zero exit status 128.

Homebrew logs show the same error, although they don't run this exact command.

Just to note that the MacPorts build actually runs python3.11 -m build --wheel --no-isolation --outdir ..., but it's all the same.

harens commented 1 year ago

Actually there is a much deeper problem here: the source distribution contains nothing

I think it should be gitlint_core, not gitlint.

ofek commented 1 year ago

I see, if you remove this everything will work https://github.com/jorisroovers/gitlint/blob/014bfaeac8c75bf0a06b55e533a39a79b3c69cd3/pyproject.toml#L48-L49

Currently the metadata hook does not read the PKG-INFO file from the source distribution and tries the same Git logic. Can you please open an issue?

webknjaz commented 1 year ago

Building from github tarball:

I mentioned how to fix this in https://github.com/jorisroovers/gitlint/pull/410#issuecomment-1462049126. TL;DR — set up .git_archival.txt properly.

jorisroovers commented 1 year ago

Since 0.19.1 is out the door with the fix, I'm closing this issue. Development of 0.20.0 is tracked in #462

Thanks for all the help!