gitlint and gitlint-core v0.19.0 fail to build from sdist

[jorisroovers]

This issue was first discovered and discussed in #367.

To reproduce for gitlint-core:

> wget https://files.pythonhosted.org/packages/29/59/c317a4a39657a4a76cd7dc0c9db4147cca57f13ebe11a469d8c6c0f88715/gitlint_core-0.19.0.tar.gz
> tar xvf gitlint_core-0.19.0.tar.gz
> cd gitlint_core-0.19.0/
> hatch build -t wheel                                                                                                                 
[wheel]
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/vscode/.local/share/hatch/env/virtual/gitlint-core/ovOkWHgP/gitlint-core-build/lib/python3.11/site-packages/hatchling/__main__.py", line 6, in <module>
    sys.exit(hatchling())
             ^^^^^^^^^^^
  File "/home/vscode/.local/share/hatch/env/virtual/gitlint-core/ovOkWHgP/gitlint-core-build/lib/python3.11/site-packages/hatchling/cli/__init__.py", line 26, in hatchling
    command(**kwargs)
  File "/home/vscode/.local/share/hatch/env/virtual/gitlint-core/ovOkWHgP/gitlint-core-build/lib/python3.11/site-packages/hatchling/cli/build/__init__.py", line 75, in build_impl
    for artifact in builder.build(
  File "/home/vscode/.local/share/hatch/env/virtual/gitlint-core/ovOkWHgP/gitlint-core-build/lib/python3.11/site-packages/hatchling/builders/plugin/interface.py", line 93, in build
    self.metadata.validate_fields()
  File "/home/vscode/.local/share/hatch/env/virtual/gitlint-core/ovOkWHgP/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 243, in validate_fields
    _ = self.version
        ^^^^^^^^^^^^
  File "/home/vscode/.local/share/hatch/env/virtual/gitlint-core/ovOkWHgP/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 128, in version
    self._version = self._get_version()
                    ^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/share/hatch/env/virtual/gitlint-core/ovOkWHgP/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 222, in _get_version
    core_metadata = self.core
                    ^^^^^^^^^
  File "/home/vscode/.local/share/hatch/env/virtual/gitlint-core/ovOkWHgP/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 174, in core
    metadata_hook.update(self.core_raw_metadata)
  File "/home/vscode/.local/share/hatch/env/virtual/gitlint-core/ovOkWHgP/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/metadata_hook.py", line 45, in update
    urls[key] = formatter.format(url)
                ^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.asdf/installs/python/3.11.0/lib/python3.11/string.py", line 190, in format
    return self.vformat(format_string, args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/share/hatch/env/virtual/gitlint-core/ovOkWHgP/gitlint-core-build/lib/python3.11/site-packages/hatchling/utils/context.py", line 127, in vformat
    result, _ = self._vformat(format_string, args, kwargs, used_args, 10)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.asdf/installs/python/3.11.0/lib/python3.11/string.py", line 247, in _vformat
    result.append(self.format_field(obj, format_spec))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/share/hatch/env/virtual/gitlint-core/ovOkWHgP/gitlint-core-build/lib/python3.11/site-packages/hatchling/utils/context.py", line 145, in format_field
    return self.__formatters[formatter](value, data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/share/hatch/env/virtual/gitlint-core/ovOkWHgP/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/metadata_hook.py", line 39, in <lambda>
    'commit_hash': lambda *args: vcs_utils.get_commit_hash(self.root),
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/share/hatch/env/virtual/gitlint-core/ovOkWHgP/gitlint-core-build/lib/python3.11/site-packages/hatch_vcs/vcs_utils.py", line 10, in get_commit_hash
    return subprocess.check_output(['git', 'rev-parse', 'HEAD'], cwd=root).decode('utf-8').strip()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.asdf/installs/python/3.11.0/lib/python3.11/subprocess.py", line 465, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.asdf/installs/python/3.11.0/lib/python3.11/subprocess.py", line 569, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'rev-parse', 'HEAD']' returned non-zero exit status 128

This issue is caused by a bug in hatch-vcs metadata hook. I’ll be opening a bug with hatch-vcs for this soon.

For gitlint-core, it’s this line that’s problematic (thanks for pointing this out @ofek):

https://github.com/jorisroovers/gitlint/blob/014bfaeac8c75bf0a06b55e533a39a79b3c69cd3/gitlint-core/pyproject.toml#L70

The same issue exists for the gitlint package, caused by these 2 lines:

https://github.com/jorisroovers/gitlint/blob/014bfaeac8c75bf0a06b55e533a39a79b3c69cd3/pyproject.toml#L48-L49

The fix in both cases is to remove these lines from the respective pyproject.toml files.

However, after doing so, I ran into another issue with gitlint-core:

> hatch build -t wheel                                                                                                                   
[wheel]
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/vscode/.local/share/hatch/env/virtual/gitlint-core/ovOkWHgP/gitlint-core-build/lib/python3.11/site-packages/hatchling/__main__.py", line 6, in <module>
    sys.exit(hatchling())
             ^^^^^^^^^^^
  File "/home/vscode/.local/share/hatch/env/virtual/gitlint-core/ovOkWHgP/gitlint-core-build/lib/python3.11/site-packages/hatchling/cli/__init__.py", line 26, in hatchling
    command(**kwargs)
  File "/home/vscode/.local/share/hatch/env/virtual/gitlint-core/ovOkWHgP/gitlint-core-build/lib/python3.11/site-packages/hatchling/cli/build/__init__.py", line 75, in build_impl
    for artifact in builder.build(
  File "/home/vscode/.local/share/hatch/env/virtual/gitlint-core/ovOkWHgP/gitlint-core-build/lib/python3.11/site-packages/hatchling/builders/plugin/interface.py", line 93, in build
    self.metadata.validate_fields()
  File "/home/vscode/.local/share/hatch/env/virtual/gitlint-core/ovOkWHgP/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 244, in validate_fields
    self.core.validate_fields()
  File "/home/vscode/.local/share/hatch/env/virtual/gitlint-core/ovOkWHgP/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 1326, in validate_fields
    getattr(self, attribute)
  File "/home/vscode/.local/share/hatch/env/virtual/gitlint-core/ovOkWHgP/gitlint-core-build/lib/python3.11/site-packages/hatchling/metadata/core.py", line 509, in readme
    raise OSError(message)
OSError: Readme file does not exist: README.md

This is caused by the fact that in the source code, gitlint-core/README.md is a symlink to gitlint’s main README.md file (same for the gitlint-core/LICENSE file).

When creating the gitlint-core sdist, hatch includes the gitlint-core/README.md as a symlink file and not as the actual README.md the symlink is pointing to. When then running hatch build -t wheel on the sdist, hatch errors out trying to read the contents of the gitlint-core/README.md symlink file (now pointing nowhere).

This is easily resolved by replacing the symlink by the actual README.md file in gitlint-core in the source code.

I’ll be submitting a fix for this shortly.

CC: @ofek, @harens

[jorisroovers]

@harens This should be fixed now - I just tried a hatch build -t wheel on the sdist of the latest dev builds and it worked.

Can you confirm? Latest dev builds: https://pypi.org/project/gitlint/0.19.1.dev2/ https://pypi.org/project/gitlint-core/0.19.1.dev2/

[webknjaz]

I was just thinking about this: https://github.com/jorisroovers/gitlint/pull/418/files#r1131062741

@jorisroovers TL;DR to have a proper smoke-test, switch to invoking python -m build (without CLI args! it's important). It'll still use hatch's PEP 517 backend. This would build an sdist from Git checkout, followed by building a wheel from said sdist rather than from Git. Additionally, you can make use of https://github.com/re-actors/checkout-python-sdist to run testing from sdist and not Git. This takes smoke-testing availability for the downstream packaging since they often use sdist as the source of truth.

[sigmavirus24]

@webknjaz would you be up for sending a PR to use that?

[webknjaz]

Which part? using pypa/build? or testing from sdist? The latter would probably require some agreement and restructuring, while the former is straightforward.

[ofek]

There is also this which I've heard good things about (and of course it's Hynek) https://github.com/hynek/build-and-inspect-python-package

[webknjaz]

@ofek yes, I know about that action, but don't use it myself — it's too coupled for my taste.

FWIW I've made the small version of the PR: https://github.com/jorisroovers/gitlint/pull/463. https://github.com/re-actors/checkout-python-sdist can always be added later.

[harens]

@harens This should be fixed now - I just tried a hatch build -t wheel on the sdist of the latest dev builds and it worked.

Can you confirm? Latest dev builds: https://pypi.org/project/gitlint/0.19.1.dev2/ https://pypi.org/project/gitlint-core/0.19.1.dev2/

Seems to work fine, so I'll update the MacPorts portfile in the next stable release. Thanks for fixing @jorisroovers 👍

[jorisroovers]

Seems to work fine, so I'll update the MacPorts portfile in the next stable release. Thanks for fixing @jorisroovers 👍

Great. I'll try to release 0.19.1 with this fix later today so we can get the homebrew package out for 0.19.x and then look at the other suggestions afterwards.

[webknjaz]

@jorisroovers I recommend releasing after merging #463 so that the fix is validated in the CI, not just locally.

[jorisroovers]

@jorisroovers I recommend releasing after merging #463 so that the fix is validated in the CI, not just locally.

Ack. I'll do it tomorrow morning then (CET), rather than trying to rush it out tonight and inevitably make a mistake.

[jorisroovers]

@harens just published 0.19.1! Can you retry?

Thanks!

[harens]

Already done :)

https://github.com/macports/macports-ports/commit/3243c9c12b8bd933a4ba4c78dd85cd6a86e0cb03

https://github.com/Homebrew/homebrew-core/pull/125315

[jorisroovers]

Awesome, thanks!

I created separate issues for everything else that was suggested here - thanks everyone. Closing this one!