search

jelmer / dulwich

Pure-Python Git implementation
https://www.dulwich.io/
Other
2.07k stars 402 forks source link

Using Apple-Git breaks make check in dulwich #1188

Open kevinhendricks opened 1 year ago

kevinhendricks commented 1 year ago

FYI, I was testing some changes and I was playing with dulwich make check to verify that I broke nothing and came across the following irregularity:

Using straight dulwich master from today (and all recent earlier ones I have tested), If I use Apple's installed /usr/bin/git (ie. Apple-Git that comes with Xcode) to run make check there are 40 errors and 7 fails all related to tests that use "fast-import". This is with the very latest versions of Xcode on Monterrey and Ventura versions of macOS.

I tried with my Manjaro linux vm and all worked like a charm.

So I grabbed the latest real git source release tag and built it on my macOS machine and make check now passes all tests (skipping 82 with 1 expected fail).

So something related to Apple's supplied git has broken something related to "fast import" which I have been unable to track down (and frankly it is not worth any extra time). IMHO, Apple should not be forking tools like git just to make it work better with its horrible Xcode.

So perhaps for macOS users of dulwich interested in testing you could mention in your test docs or in a readme that they must use MacPorts git or Homebrew git or simply tell them to build git themselves. But not to use the built in git (Apple-Git from the Xcode tools) for running make check.

Hope this helps.

Kevin

jelmer commented 1 year ago

Can you provide the error output?

kevinhendricks commented 1 year ago

Will do. Most are KeyErrors for missing b'Head' and others are revparse head not returning an sha. It is almost as if the fast-import of the testdata repo .export files is broken.

Using git from git/git passes all tests.

kevinhendricks commented 1 year ago

Here is the log from running make check using Apple-Git. With normal git, everything passes:

kbhend@MacBook-Pro Desktop % which git
/usr/bin/git

kbhend@MacBook-Pro Desktop % git --version
git version 2.39.2 (Apple Git-143)

kbhend@MacBook-Pro Desktop % which python3
/Library/Frameworks/Python.framework/Versions/3.9/bin/python3

kbhend@MacBook-Pro Desktop % python3 --version
Python 3.9.8

kbhend@MacBook-Pro Desktop % mkdir testing
kbhend@MacBook-Pro Desktop % cd testing
kbhend@MacBook-Pro testing % git clone https://github.com/jelmer/dulwich.git
Cloning into 'dulwich'...
remote: Enumerating objects: 27168, done.
remote: Counting objects: 100% (64/64), done.
remote: Compressing objects: 100% (63/63), done.
remote: Total 27168 (delta 7), reused 56 (delta 1), pack-reused 27104
Receiving objects: 100% (27168/27168), 15.51 MiB | 6.10 MiB/s, done.
Resolving deltas: 100% (17772/17772), done.

kbhend@MacBook-Pro testing % cd dulwich
kbhend@MacBook-Pro dulwich % make check

python3 setup.py build
running build
running build_ext
building 'dulwich._objects' extension
creating build
creating build/temp.macosx-10.9-x86_64-3.9
creating build/temp.macosx-10.9-x86_64-3.9/dulwich
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -g -I/Library/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c dulwich/_objects.c -o build/temp.macosx-10.9-x86_64-3.9/dulwich/_objects.o
creating build/lib.macosx-10.9-x86_64-3.9
creating build/lib.macosx-10.9-x86_64-3.9/dulwich
gcc -bundle -undefined dynamic_lookup -g build/temp.macosx-10.9-x86_64-3.9/dulwich/_objects.o -o build/lib.macosx-10.9-x86_64-3.9/dulwich/_objects.cpython-39-darwin.so
building 'dulwich._pack' extension
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -g -I/Library/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c dulwich/_pack.c -o build/temp.macosx-10.9-x86_64-3.9/dulwich/_pack.o
gcc -bundle -undefined dynamic_lookup -g build/temp.macosx-10.9-x86_64-3.9/dulwich/_pack.o -o build/lib.macosx-10.9-x86_64-3.9/dulwich/_pack.cpython-39-darwin.so
building 'dulwich._diff_tree' extension
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -g -I/Library/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c dulwich/_diff_tree.c -o build/temp.macosx-10.9-x86_64-3.9/dulwich/_diff_tree.o
gcc -bundle -undefined dynamic_lookup -g build/temp.macosx-10.9-x86_64-3.9/dulwich/_diff_tree.o -o build/lib.macosx-10.9-x86_64-3.9/dulwich/_diff_tree.cpython-39-darwin.so
python3 setup.py build_ext -i
running build_ext
copying build/lib.macosx-10.9-x86_64-3.9/dulwich/_objects.cpython-39-darwin.so -> dulwich
copying build/lib.macosx-10.9-x86_64-3.9/dulwich/_pack.cpython-39-darwin.so -> dulwich
copying build/lib.macosx-10.9-x86_64-3.9/dulwich/_diff_tree.cpython-39-darwin.so -> dulwich

PYTHONHASHSEED=random PYTHONPATH=/Users/kbhend/Desktop/testing/dulwich python3 -m unittest  dulwich.tests.test_suite
.........................................................................................................s.......................................................................................ss......................................................................sssssssssssss.s......................................s......................s..........................................................................................................................................................................................................................................................................................................................................................................................................................................s..................................................ss..............................................................127.0.0.1 - - [28/Jun/2023 17:40:24] "GET /info/refs?service=git-upload-pack HTTP/1.1" 200 285
counting objects: 2, done.
127.0.0.1 - - [28/Jun/2023 17:40:24] "POST /git-upload-pack HTTP/1.1" 200 258
copied 1 pack entries 1/2
.127.0.0.1 - - [28/Jun/2023 17:40:24] "GET /info/refs?service=git-receive-pack HTTP/1.1" 200 195
127.0.0.1 - - [28/Jun/2023 17:40:24] "POST /git-receive-pack HTTP/1.1" 200 52
Push to http://localhost:50570 successful.
Ref refs/heads/master updated
..............................ss...........................................................................................s.........................................................s............................................./Users/kbhend/Desktop/testing/dulwich/dulwich/refs.py:1281: UserWarning: ref refs/heads/blah points at non-present sha 1234566789123456789123567891234657373833
  warnings.warn(
.........................................................................................................x........................................................s./Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/warnings.py:186: ResourceWarning: unclosed <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 50575)>
  filters.remove(item)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
........EEEEEEEE.........EEEEEEEE.........EEEEEEEE.........EEEEEsE.s...ssEEEEEEEEE..F........E..........F.............F................ssFsss.ss.......F.............F.............F......sss..sssssssssssssssssssssssssssssssssssssssss
======================================================================
ERROR: test_send_pack (dulwich.tests.compat.test_client.DulwichHttpClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 94, in test_send_pack
    self._do_send_pack()
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 86, in _do_send_pack
    del sendrefs[b"HEAD"]
KeyError: b'HEAD'

======================================================================
ERROR: test_send_pack_from_shallow_clone (dulwich.tests.compat.test_client.DulwichHttpClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 123, in test_send_pack_from_shallow_clone
    tree_id = local[local.head()].tree
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 636, in head
    return self.refs[b"HEAD"]
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/refs.py", line 325, in __getitem__
    raise KeyError(name)
KeyError: b'HEAD'

======================================================================
ERROR: test_send_pack_multiple_errors (dulwich.tests.compat.test_client.DulwichHttpClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 206, in test_send_pack_multiple_errors
    sendrefs, gen_pack = self.compute_send(src)
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 178, in compute_send
    del sendrefs[b"HEAD"]
KeyError: b'HEAD'

======================================================================
ERROR: test_send_pack_nothing_to_send (dulwich.tests.compat.test_client.DulwichHttpClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 98, in test_send_pack_nothing_to_send
    self._do_send_pack()
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 86, in _do_send_pack
    del sendrefs[b"HEAD"]
KeyError: b'HEAD'

======================================================================
ERROR: test_send_pack_one_error (dulwich.tests.compat.test_client.DulwichHttpClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 186, in test_send_pack_one_error
    sendrefs, gen_pack = self.compute_send(src)
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 178, in compute_send
    del sendrefs[b"HEAD"]
KeyError: b'HEAD'

======================================================================
ERROR: test_send_remove_branch (dulwich.tests.compat.test_client.DulwichHttpClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 317, in test_send_remove_branch
    sendrefs = dict(dest.refs)
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/refs.py", line 325, in __getitem__
    raise KeyError(name)
KeyError: b'HEAD'

======================================================================
ERROR: test_send_without_report_status (dulwich.tests.compat.test_client.DulwichHttpClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 146, in test_send_without_report_status
    del sendrefs[b"HEAD"]
KeyError: b'HEAD'

======================================================================
ERROR: test_archive (dulwich.tests.compat.test_client.DulwichMockSSHClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 219, in test_archive
    c.archive(self._build_path("/server_new.export"), b"HEAD", f.write)
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/client.py", line 1223, in archive
    progress(data)
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_send_pack (dulwich.tests.compat.test_client.DulwichMockSSHClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 94, in test_send_pack
    self._do_send_pack()
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 86, in _do_send_pack
    del sendrefs[b"HEAD"]
KeyError: b'HEAD'

======================================================================
ERROR: test_send_pack_from_shallow_clone (dulwich.tests.compat.test_client.DulwichMockSSHClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 123, in test_send_pack_from_shallow_clone
    tree_id = local[local.head()].tree
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 636, in head
    return self.refs[b"HEAD"]
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/refs.py", line 325, in __getitem__
    raise KeyError(name)
KeyError: b'HEAD'

======================================================================
ERROR: test_send_pack_multiple_errors (dulwich.tests.compat.test_client.DulwichMockSSHClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 206, in test_send_pack_multiple_errors
    sendrefs, gen_pack = self.compute_send(src)
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 178, in compute_send
    del sendrefs[b"HEAD"]
KeyError: b'HEAD'

======================================================================
ERROR: test_send_pack_nothing_to_send (dulwich.tests.compat.test_client.DulwichMockSSHClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 98, in test_send_pack_nothing_to_send
    self._do_send_pack()
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 86, in _do_send_pack
    del sendrefs[b"HEAD"]
KeyError: b'HEAD'

======================================================================
ERROR: test_send_pack_one_error (dulwich.tests.compat.test_client.DulwichMockSSHClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 186, in test_send_pack_one_error
    sendrefs, gen_pack = self.compute_send(src)
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 178, in compute_send
    del sendrefs[b"HEAD"]
KeyError: b'HEAD'

======================================================================
ERROR: test_send_remove_branch (dulwich.tests.compat.test_client.DulwichMockSSHClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 317, in test_send_remove_branch
    sendrefs = dict(dest.refs)
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/refs.py", line 325, in __getitem__
    raise KeyError(name)
KeyError: b'HEAD'

======================================================================
ERROR: test_send_without_report_status (dulwich.tests.compat.test_client.DulwichMockSSHClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 146, in test_send_without_report_status
    del sendrefs[b"HEAD"]
KeyError: b'HEAD'

======================================================================
ERROR: test_archive (dulwich.tests.compat.test_client.DulwichSubprocessClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 219, in test_archive
    c.archive(self._build_path("/server_new.export"), b"HEAD", f.write)
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/client.py", line 1223, in archive
    progress(data)
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_send_pack (dulwich.tests.compat.test_client.DulwichSubprocessClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 94, in test_send_pack
    self._do_send_pack()
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 86, in _do_send_pack
    del sendrefs[b"HEAD"]
KeyError: b'HEAD'

======================================================================
ERROR: test_send_pack_from_shallow_clone (dulwich.tests.compat.test_client.DulwichSubprocessClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 123, in test_send_pack_from_shallow_clone
    tree_id = local[local.head()].tree
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 636, in head
    return self.refs[b"HEAD"]
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/refs.py", line 325, in __getitem__
    raise KeyError(name)
KeyError: b'HEAD'

======================================================================
ERROR: test_send_pack_multiple_errors (dulwich.tests.compat.test_client.DulwichSubprocessClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 206, in test_send_pack_multiple_errors
    sendrefs, gen_pack = self.compute_send(src)
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 178, in compute_send
    del sendrefs[b"HEAD"]
KeyError: b'HEAD'

======================================================================
ERROR: test_send_pack_nothing_to_send (dulwich.tests.compat.test_client.DulwichSubprocessClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 98, in test_send_pack_nothing_to_send
    self._do_send_pack()
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 86, in _do_send_pack
    del sendrefs[b"HEAD"]
KeyError: b'HEAD'

======================================================================
ERROR: test_send_pack_one_error (dulwich.tests.compat.test_client.DulwichSubprocessClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 186, in test_send_pack_one_error
    sendrefs, gen_pack = self.compute_send(src)
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 178, in compute_send
    del sendrefs[b"HEAD"]
KeyError: b'HEAD'

======================================================================
ERROR: test_send_remove_branch (dulwich.tests.compat.test_client.DulwichSubprocessClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 317, in test_send_remove_branch
    sendrefs = dict(dest.refs)
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/refs.py", line 325, in __getitem__
    raise KeyError(name)
KeyError: b'HEAD'

======================================================================
ERROR: test_send_without_report_status (dulwich.tests.compat.test_client.DulwichSubprocessClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 146, in test_send_without_report_status
    del sendrefs[b"HEAD"]
KeyError: b'HEAD'

======================================================================
ERROR: test_archive (dulwich.tests.compat.test_client.DulwichTCPClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 219, in test_archive
    c.archive(self._build_path("/server_new.export"), b"HEAD", f.write)
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/client.py", line 1223, in archive
    progress(data)
TypeError: 'NoneType' object is not callable

======================================================================
ERROR: test_send_pack (dulwich.tests.compat.test_client.DulwichTCPClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 94, in test_send_pack
    self._do_send_pack()
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 86, in _do_send_pack
    del sendrefs[b"HEAD"]
KeyError: b'HEAD'

======================================================================
ERROR: test_send_pack_from_shallow_clone (dulwich.tests.compat.test_client.DulwichTCPClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 123, in test_send_pack_from_shallow_clone
    tree_id = local[local.head()].tree
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 636, in head
    return self.refs[b"HEAD"]
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/refs.py", line 325, in __getitem__
    raise KeyError(name)
KeyError: b'HEAD'

======================================================================
ERROR: test_send_pack_multiple_errors (dulwich.tests.compat.test_client.DulwichTCPClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 206, in test_send_pack_multiple_errors
    sendrefs, gen_pack = self.compute_send(src)
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 178, in compute_send
    del sendrefs[b"HEAD"]
KeyError: b'HEAD'

======================================================================
ERROR: test_send_pack_nothing_to_send (dulwich.tests.compat.test_client.DulwichTCPClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 98, in test_send_pack_nothing_to_send
    self._do_send_pack()
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 86, in _do_send_pack
    del sendrefs[b"HEAD"]
KeyError: b'HEAD'

======================================================================
ERROR: test_send_pack_one_error (dulwich.tests.compat.test_client.DulwichTCPClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 186, in test_send_pack_one_error
    sendrefs, gen_pack = self.compute_send(src)
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 178, in compute_send
    del sendrefs[b"HEAD"]
KeyError: b'HEAD'

======================================================================
ERROR: test_send_without_report_status (dulwich.tests.compat.test_client.DulwichTCPClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_client.py", line 146, in test_send_without_report_status
    del sendrefs[b"HEAD"]
KeyError: b'HEAD'

======================================================================
ERROR: test_all_objects (dulwich.tests.compat.test_repository.InitNewWorkingDirectoryTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_repository.py", line 222, in setUp
    self._repo = Repo._init_new_working_directory(
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 1744, in _init_new_working_directory
    f.write(main_repo.head() + b"\n")
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 636, in head
    return self.refs[b"HEAD"]
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/refs.py", line 325, in __getitem__
    raise KeyError(name)
KeyError: b'HEAD'

======================================================================
ERROR: test_bare (dulwich.tests.compat.test_repository.InitNewWorkingDirectoryTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_repository.py", line 222, in setUp
    self._repo = Repo._init_new_working_directory(
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 1744, in _init_new_working_directory
    f.write(main_repo.head() + b"\n")
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 636, in head
    return self.refs[b"HEAD"]
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/refs.py", line 325, in __getitem__
    raise KeyError(name)
KeyError: b'HEAD'

======================================================================
ERROR: test_git_worktree_config (dulwich.tests.compat.test_repository.InitNewWorkingDirectoryTestCase)
Test that git worktree config parsing matches the git CLI's behavior.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_repository.py", line 222, in setUp
    self._repo = Repo._init_new_working_directory(
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 1744, in _init_new_working_directory
    f.write(main_repo.head() + b"\n")
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 636, in head
    return self.refs[b"HEAD"]
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/refs.py", line 325, in __getitem__
    raise KeyError(name)
KeyError: b'HEAD'

======================================================================
ERROR: test_git_worktree_list (dulwich.tests.compat.test_repository.InitNewWorkingDirectoryTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_repository.py", line 222, in setUp
    self._repo = Repo._init_new_working_directory(
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 1744, in _init_new_working_directory
    f.write(main_repo.head() + b"\n")
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 636, in head
    return self.refs[b"HEAD"]
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/refs.py", line 325, in __getitem__
    raise KeyError(name)
KeyError: b'HEAD'

======================================================================
ERROR: test_head (dulwich.tests.compat.test_repository.InitNewWorkingDirectoryTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_repository.py", line 222, in setUp
    self._repo = Repo._init_new_working_directory(
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 1744, in _init_new_working_directory
    f.write(main_repo.head() + b"\n")
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 636, in head
    return self.refs[b"HEAD"]
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/refs.py", line 325, in __getitem__
    raise KeyError(name)
KeyError: b'HEAD'

======================================================================
ERROR: test_head_equality (dulwich.tests.compat.test_repository.InitNewWorkingDirectoryTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_repository.py", line 222, in setUp
    self._repo = Repo._init_new_working_directory(
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 1744, in _init_new_working_directory
    f.write(main_repo.head() + b"\n")
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 636, in head
    return self.refs[b"HEAD"]
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/refs.py", line 325, in __getitem__
    raise KeyError(name)
KeyError: b'HEAD'

======================================================================
ERROR: test_loose_objects (dulwich.tests.compat.test_repository.InitNewWorkingDirectoryTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_repository.py", line 222, in setUp
    self._repo = Repo._init_new_working_directory(
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 1744, in _init_new_working_directory
    f.write(main_repo.head() + b"\n")
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 636, in head
    return self.refs[b"HEAD"]
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/refs.py", line 325, in __getitem__
    raise KeyError(name)
KeyError: b'HEAD'

======================================================================
ERROR: test_packed_objects (dulwich.tests.compat.test_repository.InitNewWorkingDirectoryTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_repository.py", line 222, in setUp
    self._repo = Repo._init_new_working_directory(
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 1744, in _init_new_working_directory
    f.write(main_repo.head() + b"\n")
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 636, in head
    return self.refs[b"HEAD"]
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/refs.py", line 325, in __getitem__
    raise KeyError(name)
KeyError: b'HEAD'

======================================================================
ERROR: test_refs (dulwich.tests.compat.test_repository.InitNewWorkingDirectoryTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_repository.py", line 222, in setUp
    self._repo = Repo._init_new_working_directory(
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 1744, in _init_new_working_directory
    f.write(main_repo.head() + b"\n")
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/repo.py", line 636, in head
    return self.refs[b"HEAD"]
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/refs.py", line 325, in __getitem__
    raise KeyError(name)
KeyError: b'HEAD'

======================================================================
ERROR: test_head_equality (dulwich.tests.compat.test_repository.WorkingTreeTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_repository.py", line 156, in test_head_equality
    self._repo.refs[b"HEAD"], self._mainworktree_repo.refs[b"HEAD"]
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/refs.py", line 325, in __getitem__
    raise KeyError(name)
KeyError: b'HEAD'

======================================================================
FAIL: test_head (dulwich.tests.compat.test_repository.ObjectStoreTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/test_repository.py", line 65, in test_head
    hex_to_sha(head_sha)
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/objects.py", line 119, in hex_to_sha
    assert len(hex) == 40, "Incorrect length of hexsha: %s" % hex
AssertionError: Incorrect length of hexsha: b'HEAD'

======================================================================
FAIL: test_lsremote_from_dulwich (dulwich.tests.compat.test_server.GitServerSideBand64kTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/server_utils.py", line 165, in test_lsremote_from_dulwich
    self.assertEqual(len(o.split(b"\n")), 4)
AssertionError: 3 != 4

======================================================================
FAIL: test_lsremote_from_dulwich (dulwich.tests.compat.test_server.GitServerTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/server_utils.py", line 165, in test_lsremote_from_dulwich
    self.assertEqual(len(o.split(b"\n")), 4)
AssertionError: 3 != 4

======================================================================
FAIL: test_lsremote_from_dulwich (dulwich.tests.compat.test_web.DumbWebTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/server_utils.py", line 165, in test_lsremote_from_dulwich
    self.assertEqual(len(o.split(b"\n")), 4)
AssertionError: 3 != 4

======================================================================
FAIL: test_lsremote_from_dulwich (dulwich.tests.compat.test_web.SmartWebSideBand64kNoDoneTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/server_utils.py", line 165, in test_lsremote_from_dulwich
    self.assertEqual(len(o.split(b"\n")), 4)
AssertionError: 3 != 4

======================================================================
FAIL: test_lsremote_from_dulwich (dulwich.tests.compat.test_web.SmartWebSideBand64kTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/server_utils.py", line 165, in test_lsremote_from_dulwich
    self.assertEqual(len(o.split(b"\n")), 4)
AssertionError: 3 != 4

======================================================================
FAIL: test_lsremote_from_dulwich (dulwich.tests.compat.test_web.SmartWebTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/kbhend/Desktop/testing/dulwich/dulwich/tests/compat/server_utils.py", line 165, in test_lsremote_from_dulwich
    self.assertEqual(len(o.split(b"\n")), 4)
AssertionError: 3 != 4

----------------------------------------------------------------------
Ran 1507 tests in 92.835s

FAILED (failures=7, errors=40, skipped=82, expected failures=1)
make: *** [check] Error 1
kbhend@MacBook-Pro dulwich %
kevinhendricks commented 1 year ago

With normal git:

kbhend@MacBook-Pro Desktop % which git
/usr/local/bin/git

kbhend@MacBook-Pro Desktop % git --version
git version 2.39.3

Same python3, all tests pass:

PYTHONHASHSEED=random PYTHONPATH=/Users/kbhend/Desktop/testing/dulwich python3 -m unittest  dulwich.tests.test_suite
.........................................................................................................s.......................................................................................ss......................................................................sssssssssssss.s......................................s......................s..........................................................................................................................................................................................................................................................................................................................................................................................................................................s..................................................ss..............................................................127.0.0.1 - - [28/Jun/2023 17:49:50] "GET /info/refs?service=git-upload-pack HTTP/1.1" 200 285
counting objects: 2, done.
127.0.0.1 - - [28/Jun/2023 17:49:50] "POST /git-upload-pack HTTP/1.1" 200 258
copied 1 pack entries 1/2
.127.0.0.1 - - [28/Jun/2023 17:49:51] "GET /info/refs?service=git-receive-pack HTTP/1.1" 200 195
127.0.0.1 - - [28/Jun/2023 17:49:51] "POST /git-receive-pack HTTP/1.1" 200 52
Push to http://localhost:51039 successful.
Ref refs/heads/master updated
..............................ss...........................................................................................s.........................................................s............................................./Users/kbhend/Desktop/testing/dulwich/dulwich/refs.py:1281: UserWarning: ref refs/heads/blah points at non-present sha 1234566789123456789123567891234657373833
  warnings.warn(
.........................................................................................................x........................................................s./Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/warnings.py:186: ResourceWarning: unclosed <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 51044)>
  filters.remove(item)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
................................................................s..s...ss..............................................................ss.sss.ss..........................................sss..sssssssssssssssssssssssssssssssssssssssss
----------------------------------------------------------------------
Ran 1507 tests in 87.999s

OK (skipped=82, expected failures=1)
jelmer commented 1 year ago

Thanks; nothing immediately jumps out that might be the cause of the breakage. There are no relevant changes between 2.39.2 and 2.39.3 that I can see:

https://github.com/git/git/blob/master/Documentation/RelNotes/2.39.3.txt

And the Apple-git specific changes also don't appear to be relevant:

https://github.com/jeremyhu/git/commits/master

kevinhendricks commented 1 year ago

Yes, I looked for about an hour, even managed to add progress and checkpoint commands to the testdata/repos/ .export files and checked that "progress done" was emitted by apple-git when it did the fast-import. It was. But the tests still failed. I even modded the tests/compat/utils.py file to not do a clean up and manually printed out the repo.path in /var/tmp so that I could look there and all appeared to be fine.

I am not sure why this is happening. But thought you might want to warn other mac users about the need to use real git when running the dulwich make check just to prevent incorrect bug reports.