search

arthaud / git-dumper

A tool to dump a git repository from a website
MIT License
1.88k stars 253 forks source link

AttributeError: 'Index' object has no attribute 'iterblobs' #27

Closed peter17 closed 2 years ago

peter17 commented 2 years ago

Hi,

Tanks for your tool, it works very well, in most cases. However, I had the following error:

$ gitdumper.py https://(REDACTED)/.git/ repository
[-] Testing https://(REDACTED)/.git/HEAD [200]
[-] Testing https://(REDACTED)/.git/ [403]
[-] Fetching common files
[-] Fetching https://(REDACTED)/.git/COMMIT_EDITMSG [404]
[-] https://(REDACTED)/.git/COMMIT_EDITMSG responded with status code 404
[-] Fetching https://(REDACTED)/.git/hooks/applypatch-msg.sample [200]
[-] Fetching https://(REDACTED)/.git/description [200]
[-] Fetching https://(REDACTED)/.git/hooks/pre-applypatch.sample [200]
[-] Fetching https://(REDACTED)/.git/hooks/post-receive.sample [404]
[-] https://(REDACTED)/.git/hooks/post-receive.sample responded with status code 404
[-] Fetching https://(REDACTED)/.git/hooks/post-commit.sample [404]
[-] https://(REDACTED)/.git/hooks/post-commit.sample responded with status code 404
[-] Fetching https://(REDACTED)/.git/hooks/post-update.sample [200]
[-] Fetching https://(REDACTED)/.git/hooks/pre-commit.sample [200]
[-] Fetching https://(REDACTED)/.gitignore [404]
[-] https://(REDACTED)/.gitignore responded with status code 404
[-] Fetching https://(REDACTED)/.git/hooks/commit-msg.sample [200]
[-] Fetching https://(REDACTED)/.git/hooks/pre-receive.sample [404]
[-] https://(REDACTED)/.git/hooks/pre-receive.sample responded with status code 404
[-] Fetching https://(REDACTED)/.git/hooks/prepare-commit-msg.sample [200]
[-] Fetching https://(REDACTED)/.git/hooks/pre-rebase.sample [200]
[-] Fetching https://(REDACTED)/.git/objects/info/packs [404]
[-] https://(REDACTED)/.git/objects/info/packs responded with status code 404
[-] Fetching https://(REDACTED)/.git/info/exclude [200]
[-] Fetching https://(REDACTED)/.git/hooks/pre-push.sample [200]
[-] Fetching https://(REDACTED)/.git/hooks/update.sample [200]
[-] Fetching https://(REDACTED)/.git/index [200]
[-] Finding refs/
[-] Fetching https://(REDACTED)/.git/info/refs [404]
[-] https://(REDACTED)/.git/info/refs responded with status code 404
[-] Fetching https://(REDACTED)/.git/config [200]
[-] Fetching https://(REDACTED)/.git/ORIG_HEAD [200]
[-] Fetching https://(REDACTED)/.git/logs/refs/stash [404]
[-] https://(REDACTED)/.git/logs/refs/stash responded with status code 404
[-] Fetching https://(REDACTED)/.git/logs/refs/remotes/origin/master [404]
[-] https://(REDACTED)/.git/logs/refs/remotes/origin/master responded with status code 404
[-] Fetching https://(REDACTED)/.git/HEAD [200]
[-] Fetching https://(REDACTED)/.git/FETCH_HEAD [200]
[-] Fetching https://(REDACTED)/.git/packed-refs [200]
[-] Fetching https://(REDACTED)/.git/logs/refs/remotes/origin/HEAD [200]
[-] Fetching https://(REDACTED)/.git/logs/refs/heads/master [404]
[-] https://(REDACTED)/.git/logs/refs/heads/master responded with status code 404
[-] Fetching https://(REDACTED)/.git/refs/heads/master [404]
[-] https://(REDACTED)/.git/refs/heads/master responded with status code 404
[-] Fetching https://(REDACTED)/.git/logs/HEAD [200]
[-] Fetching https://(REDACTED)/.git/refs/remotes/origin/HEAD [200]
[-] Fetching https://(REDACTED)/.git/refs/remotes/origin/master [404]
[-] https://(REDACTED)/.git/refs/remotes/origin/master responded with status code 404
[-] Fetching https://(REDACTED)/.git/refs/wip/wtree/refs/heads/master [404]
[-] https://(REDACTED)/.git/refs/wip/wtree/refs/heads/master responded with status code 404
[-] Fetching https://(REDACTED)/.git/refs/stash [404]
[-] https://(REDACTED)/.git/refs/stash responded with status code 404
[-] Fetching https://(REDACTED)/.git/refs/wip/index/refs/heads/master [404]
[-] https://(REDACTED)/.git/refs/wip/index/refs/heads/master responded with status code 404
[-] Fetching https://(REDACTED)/.git/refs/heads/develop [200]
[-] Fetching https://(REDACTED)/.git/refs/remotes/origin/develop [404]
[-] https://(REDACTED)/.git/refs/remotes/origin/develop responded with status code 404
[-] Fetching https://(REDACTED)/.git/logs/refs/heads/develop [200]
[-] Fetching https://(REDACTED)/.git/logs/refs/remotes/origin/develop [404]
[-] https://(REDACTED)/.git/logs/refs/remotes/origin/develop responded with status code 404
[-] Fetching https://(REDACTED)/.git/logs/refs/tags/1.0.0 [404]
[-] https://(REDACTED)/.git/logs/refs/tags/1.0.0 responded with status code 404
[-] Fetching https://(REDACTED)/.git/refs/tags/1.0.2 [404]
[-] https://(REDACTED)/.git/refs/tags/1.0.2 responded with status code 404
[-] Fetching https://(REDACTED)/.git/logs/refs/tags/1.0.1 [404]
[-] https://(REDACTED)/.git/logs/refs/tags/1.0.1 responded with status code 404
[-] Fetching https://(REDACTED)/.git/logs/refs/tags/1.0.2 [404]
[-] https://(REDACTED)/.git/logs/refs/tags/1.0.2 responded with status code 404
[-] Fetching https://(REDACTED)/.git/refs/tags/1.1.0 [404]
[-] https://(REDACTED)/.git/refs/tags/1.1.0 responded with status code 404
[-] Fetching https://(REDACTED)/.git/refs/tags/1.0.0 [404]
[-] https://(REDACTED)/.git/refs/tags/1.0.0 responded with status code 404
[-] Fetching https://(REDACTED)/.git/refs/tags/1.0.1 [404]
[-] https://(REDACTED)/.git/refs/tags/1.0.1 responded with status code 404
[-] Fetching https://(REDACTED)/.git/refs/tags/1.1.1 [404]
[-] https://(REDACTED)/.git/refs/tags/1.1.1 responded with status code 404
[-] Fetching https://(REDACTED)/.git/logs/refs/tags/1.1.0 [404]
[-] https://(REDACTED)/.git/logs/refs/tags/1.1.0 responded with status code 404
[-] Fetching https://(REDACTED)/.git/logs/refs/tags/1.1.1 [404]
[-] https://(REDACTED)/.git/logs/refs/tags/1.1.1 responded with status code 404
[-] Fetching https://(REDACTED)/.git/refs/tags/1.1.2 [404]
[-] Fetching https://(REDACTED)/.git/logs/refs/tags/1.1.2 [404]
[-] https://(REDACTED)/.git/refs/tags/1.1.2 responded with status code 404
[-] https://(REDACTED)/.git/logs/refs/tags/1.1.2 responded with status code 404
[-] Finding packs
[-] Finding objects
Traceback (most recent call last):
  File "/home/peter/bin/gitdumper.py", line 724, in <module>
    main()
  File "/home/peter/bin/gitdumper.py", line 712, in main
    fetch_git(
  File "/home/peter/bin/gitdumper.py", line 571, in fetch_git
    for entry in index.iterblobs():
AttributeError: 'Index' object has no attribute 'iterblobs'

Can you please have a look? Thanks a lot!

peter17 commented 2 years ago

NB: after some investigation, I found that it works if i install dulwich == 0.20.20. I previously had 0.20.30

I have not tried with other versions. You may want to update either your code to work with newer versions of dulwich or requirements.txt to lock it to a specific version.

bubzzzzzzzz commented 2 years ago

Hi, I am still unable to get this working please can someone help.

┌──(kali㉿kali)-[~] └─$ git-dumper http://10.129.96.151/.git ./eplison [-] Testing http://10.129.96.151/.git/HEAD [200] [-] Testing http://10.129.96.151/.git/ [403] [-] Fetching common files [-] Fetching http://10.129.96.151/.gitignore [404] [-] http://10.129.96.151/.gitignore responded with status code 404 [-] Fetching http://10.129.96.151/.git/hooks/commit-msg.sample [200] [-] Fetching http://10.129.96.151/.git/hooks/applypatch-msg.sample [200] [-] Fetching http://10.129.96.151/.git/COMMIT_EDITMSG [200] [-] Fetching http://10.129.96.151/.git/hooks/post-commit.sample [404] [-] http://10.129.96.151/.git/hooks/post-commit.sample responded with status code 404 [-] Fetching http://10.129.96.151/.git/hooks/pre-rebase.sample [200] [-] Fetching http://10.129.96.151/.git/description [200] [-] Fetching http://10.129.96.151/.git/hooks/pre-applypatch.sample [200] [-] Fetching http://10.129.96.151/.git/hooks/pre-push.sample [200] [-] Fetching http://10.129.96.151/.git/hooks/post-receive.sample [404] [-] http://10.129.96.151/.git/hooks/post-receive.sample responded with status code 404 [-] Fetching http://10.129.96.151/.git/hooks/update.sample [200] [-] Fetching http://10.129.96.151/.git/hooks/post-update.sample [200] [-] Fetching http://10.129.96.151/.git/hooks/pre-commit.sample [200] [-] Fetching http://10.129.96.151/.git/hooks/pre-receive.sample [200] [-] Fetching http://10.129.96.151/.git/objects/info/packs [404] [-] http://10.129.96.151/.git/objects/info/packs responded with status code 404 [-] Fetching http://10.129.96.151/.git/index [200] [-] Fetching http://10.129.96.151/.git/info/exclude [200] [-] Fetching http://10.129.96.151/.git/hooks/prepare-commit-msg.sample [200] [-] Finding refs/ [-] Fetching http://10.129.96.151/.git/HEAD [200] [-] Fetching http://10.129.96.151/.git/config [200] [-] Fetching http://10.129.96.151/.git/FETCH_HEAD [404] [-] http://10.129.96.151/.git/FETCH_HEAD responded with status code 404 [-] Fetching http://10.129.96.151/.git/logs/refs/stash [404] [-] http://10.129.96.151/.git/logs/refs/stash responded with status code 404 [-] Fetching http://10.129.96.151/.git/info/refs [404] [-] http://10.129.96.151/.git/info/refs responded with status code 404 [-] Fetching http://10.129.96.151/.git/logs/refs/heads/master [200] [-] Fetching http://10.129.96.151/.git/refs/heads/master [200] [-] Fetching http://10.129.96.151/.git/ORIG_HEAD [200] [-] Fetching http://10.129.96.151/.git/logs/refs/remotes/origin/HEAD [404] [-] http://10.129.96.151/.git/logs/refs/remotes/origin/HEAD responded with status code 404 [-] Fetching http://10.129.96.151/.git/logs/HEAD [200] [-] Fetching http://10.129.96.151/.git/refs/remotes/origin/HEAD [404] [-] Fetching http://10.129.96.151/.git/refs/remotes/origin/master [404] [-] http://10.129.96.151/.git/refs/remotes/origin/master responded with status code 404 [-] http://10.129.96.151/.git/refs/remotes/origin/HEAD responded with status code 404 [-] Fetching http://10.129.96.151/.git/logs/refs/remotes/origin/master [404] [-] http://10.129.96.151/.git/logs/refs/remotes/origin/master responded with status code 404 [-] Fetching http://10.129.96.151/.git/refs/stash [404] [-] http://10.129.96.151/.git/refs/stash responded with status code 404 [-] Fetching http://10.129.96.151/.git/packed-refs [404] [-] http://10.129.96.151/.git/packed-refs responded with status code 404 [-] Fetching http://10.129.96.151/.git/refs/wip/wtree/refs/heads/master [404] [-] http://10.129.96.151/.git/refs/wip/wtree/refs/heads/master responded with status code 404 [-] Fetching http://10.129.96.151/.git/refs/wip/index/refs/heads/master [404] [-] http://10.129.96.151/.git/refs/wip/index/refs/heads/master responded with status code 404 [-] Finding packs [-] Finding objects Traceback (most recent call last): File "/usr/local/bin/git-dumper", line 8, in sys.exit(main()) File "/home/kali/.local/lib/python3.9/site-packages/git_dumper.py", line 724, in main fetch_git( File "/home/kali/.local/lib/python3.9/site-packages/git_dumper.py", line 583, in fetch_git for entry in index.iterblobs(): AttributeError: 'Index' object has no attribute 'iterblobs'

git_dumper.txt

dulwich version is 0.20.20

bubzzzzzzzz commented 2 years ago

Hi, after speaking to Peter this is now working, related to dulwich version.

Fix which Peter provided: pip3 install dulwich==0.20.20 sudo pip3 install dulwich==0.20.20

Thanks

arthaud commented 2 years ago

Should be fixed with 0e68785456ff9d8f07e42feb34cc33d832879692

OxNinja commented 2 years ago

Hello, sorry to reopen this, yes #28 fixed the problem :pray:, but the pip package does not include this version of dulwich :slightly_smiling_face: If installed with pip install git-dumper, one should also pip install dulwich==0.20.20

arthaud commented 2 years ago

This should be fixed in the latest package, 1.0.6.