search

Kentzo / git-archive-all

A python script wrapper for git-archive that archives a git superproject and its submodules, if it has any. Takes into account .gitattributes
MIT License
372 stars 81 forks source link

Segment fault when output as tar.bz2 #59

Closed wuzhouhui closed 5 years ago

wuzhouhui commented 5 years ago

Reproduce steps: ./git_archive_all.py a.tar.bz2 OS: Ubuntu 18.04 Python: 2.7.15rc1

Kentzo commented 5 years ago

Hi @wuzhouhui,

What version of git-archive-all are you running? What is the version of git?

If you have Python 3.3+, please try to reproduce the problem by running it like this:

python3 -q -X faulthandler ./git_archive_all.py a.tar.bz2

wuzhouhui commented 5 years ago

Hi @wuzhouhui,

What version of git-archive-all are you running? What is the version of git?

version of git-archive-all is 1.19.0, version of git is 2.17.1

If you have Python 3.3+, please try to reproduce the problem by running it like this:

python3 -q -X faulthandler ./git_archive_all.py a.tar.bz2

I'm using Python-3.6.6 to run python3 -q -X faulthandler ./git_archive_all.py a.tar.bz2, and command output nothing. Segfault doesn't appered and a.tar.bz2 generated successfully, thanks.

Kentzo commented 5 years ago

@wuzhouhui Hmm, that's not expected :) -q -X faulthandler is supposed to give more details when segfault happens.

Is there anything unusual about the repo? Does it have submodules or .gitattributes?

wuzhouhui commented 5 years ago

@wuzhouhui Hmm, that's not expected :) -q -X faulthandler is supposed to give more details when segfault happens.

The segfault doesn't happens when I using Python 3.6.6, so I think -q -X faulthandler output nothing is normal.

Is there anything unusual about the repo? Does it have submodules or .gitattributes?

See following: That's what I did:

$ git clone https://github.com/Kentzo/git-archive-all.git
$ cd git-archive-all/
$ ./git_archive_all.py a.tar.bz2
Segmentation fault (core dumped)
$ python3.6 -q -X faulthandler ./git_archive_all.py a.tar.bz2
$ git status
On branch master
Your branch is up to date with 'origin/master'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)

    a.tar.bz2

nothing added to commit but untracked files present (use "git add" to track)
$

When segfault happens, the version of python that runs scripts is 2.7.15.rc1

wuzhouhui commented 5 years ago

Maybe my Python-2.7.15rc1 is broken.

Kentzo commented 5 years ago

Could you try 1.18.3?

wuzhouhui commented 5 years ago

Could you try 1.18.3?

The segfault doesn't happens in 1.18.3:

$ git checkout 1.18.3
$ ./git_archive_all.py a.tar.bz2
$ git status
HEAD detached at 1.18.3
Untracked files:
  (use "git add <file>..." to include in what will be committed)

    a.tar.bz2

nothing added to commit but untracked files present (use "git add" to track)

The version of python that runs script is 2.7.15rc1

Kentzo commented 5 years ago

With 1.19.0 and python 2.7, could you try python ./git_archive_all.py --dry-run -vv a.tar.bz2?

wuzhouhui commented 5 years ago

With 1.19.0 and python 2.7, could you try python ./git_archive_all.py --dry-run -vv a.tar.bz2?

$ git checkout 1.19.0
$ python2.7 ./git_archive_all.py --dry-run -vv a.tar.bz2
Output format is not explicitly set, determined format is bz2.
/home/wzh/Downloads/git-archive-all/.gitignore => a/.gitignore
/home/wzh/Downloads/git-archive-all/.travis.yml => a/.travis.yml
/home/wzh/Downloads/git-archive-all/CHANGES.rst => a/CHANGES.rst
/home/wzh/Downloads/git-archive-all/LICENSE.txt => a/LICENSE.txt
/home/wzh/Downloads/git-archive-all/MANIFEST.in => a/MANIFEST.in
/home/wzh/Downloads/git-archive-all/Makefile => a/Makefile
/home/wzh/Downloads/git-archive-all/README.rst => a/README.rst
/home/wzh/Downloads/git-archive-all/git_archive_all.py => a/git_archive_all.py
/home/wzh/Downloads/git-archive-all/requirements.txt => a/requirements.txt
/home/wzh/Downloads/git-archive-all/setup.cfg => a/setup.cfg
/home/wzh/Downloads/git-archive-all/setup.py => a/setup.py
/home/wzh/Downloads/git-archive-all/test_git_archive_all.py => a/test_git_archive_all.py
$ python2.7 ./git_archive_all.py  -vv a.tar.bz2
Output format is not explicitly set, determined format is bz2.
Segmentation fault (core dumped)
Kentzo commented 5 years ago

Interesting. So the exception happen when you try to archive it.

Could you try other formats, namely:

python2.7 ./git_archive_all.py  -vv a.tar.gz
python2.7 ./git_archive_all.py  -vv a.tar.xz
python2.7 ./git_archive_all.py  -vv a.tar
python2.7 ./git_archive_all.py  -vv a.zip

?

wuzhouhui commented 5 years ago

Interesting. So the exception happen when you try to archive it.

Could you try other formats, namely:

python2.7 ./git_archive_all.py  -vv a.tar.gz
python2.7 ./git_archive_all.py  -vv a.tar.xz
python2.7 ./git_archive_all.py  -vv a.tar
python2.7 ./git_archive_all.py  -vv a.zip

?

$ git status
HEAD detached at 1.19.0
nothing to commit, working tree clean
$ python2.7 ./git_archive_all.py -vv a.tar.gz
Output format is not explicitly set, determined format is gz.
/home/wzh/Downloads/git-archive-all/.gitignore => a/.gitignore
/home/wzh/Downloads/git-archive-all/.travis.yml => a/.travis.yml
/home/wzh/Downloads/git-archive-all/CHANGES.rst => a/CHANGES.rst
/home/wzh/Downloads/git-archive-all/LICENSE.txt => a/LICENSE.txt
/home/wzh/Downloads/git-archive-all/MANIFEST.in => a/MANIFEST.in
/home/wzh/Downloads/git-archive-all/Makefile => a/Makefile
/home/wzh/Downloads/git-archive-all/README.rst => a/README.rst
/home/wzh/Downloads/git-archive-all/git_archive_all.py => a/git_archive_all.py
/home/wzh/Downloads/git-archive-all/requirements.txt => a/requirements.txt
/home/wzh/Downloads/git-archive-all/setup.cfg => a/setup.cfg
/home/wzh/Downloads/git-archive-all/setup.py => a/setup.py
/home/wzh/Downloads/git-archive-all/test_git_archive_all.py => a/test_git_archive_all.py
$ python2.7 ./git_archive_all.py -vv a.tar.xz
Output format is not explicitly set, determined format is xz.
unknown compression type u'xz'
$ python2.7 ./git_archive_all.py -vv a.tar
Output format is not explicitly set, determined format is tar.
/home/wzh/Downloads/git-archive-all/.gitignore => a/.gitignore
/home/wzh/Downloads/git-archive-all/.travis.yml => a/.travis.yml
/home/wzh/Downloads/git-archive-all/CHANGES.rst => a/CHANGES.rst
/home/wzh/Downloads/git-archive-all/LICENSE.txt => a/LICENSE.txt
/home/wzh/Downloads/git-archive-all/MANIFEST.in => a/MANIFEST.in
/home/wzh/Downloads/git-archive-all/Makefile => a/Makefile
/home/wzh/Downloads/git-archive-all/README.rst => a/README.rst
/home/wzh/Downloads/git-archive-all/git_archive_all.py => a/git_archive_all.py
/home/wzh/Downloads/git-archive-all/requirements.txt => a/requirements.txt
/home/wzh/Downloads/git-archive-all/setup.cfg => a/setup.cfg
/home/wzh/Downloads/git-archive-all/setup.py => a/setup.py
/home/wzh/Downloads/git-archive-all/test_git_archive_all.py => a/test_git_archive_all.py
$ python2.7 ./git_archive_all.py -vv a.zip
Output format is not explicitly set, determined format is zip.
/home/wzh/Downloads/git-archive-all/.gitignore => a/.gitignore
/home/wzh/Downloads/git-archive-all/.travis.yml => a/.travis.yml
/home/wzh/Downloads/git-archive-all/CHANGES.rst => a/CHANGES.rst
/home/wzh/Downloads/git-archive-all/LICENSE.txt => a/LICENSE.txt
/home/wzh/Downloads/git-archive-all/MANIFEST.in => a/MANIFEST.in
/home/wzh/Downloads/git-archive-all/Makefile => a/Makefile
/home/wzh/Downloads/git-archive-all/README.rst => a/README.rst
/home/wzh/Downloads/git-archive-all/git_archive_all.py => a/git_archive_all.py
/home/wzh/Downloads/git-archive-all/requirements.txt => a/requirements.txt
/home/wzh/Downloads/git-archive-all/setup.cfg => a/setup.cfg
/home/wzh/Downloads/git-archive-all/setup.py => a/setup.py
/home/wzh/Downloads/git-archive-all/test_git_archive_all.py => a/test_git_archive_all.py
Kentzo commented 5 years ago

In other words it works for all formats except xz (not supported by Python 2.7) and bz2?

Kentzo commented 5 years ago

~Could you change https://github.com/Kentzo/git-archive-all/blob/b3f9c9dc8d9b7bbd91bbb9526d5c6eabf76bc16f/git_archive_all.py#L174 to archive = tarfile.open(path.abspath(output_path), mode) and check with bz2 again?~

Kentzo commented 5 years ago

Could you test git_archive_all.py from the master branch?

wuzhouhui commented 5 years ago

Could you test git_archive_all.py from the master branch?

After pull latest changes from master, ./git_archive_all.py a.tar.bz2 works well:

$ git log -1 --oneline 
c89a03d (HEAD -> master, origin/master, origin/HEAD) Do not pass compresslevel unless set by user.
$ ./git_archive_all.py a.tar.bz2
$ git status
On branch master
Your branch is up to date with 'origin/master'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)

    a.tar.bz2

nothing added to commit but untracked files present (use "git add" to track)
Kentzo commented 5 years ago

Ok, that concludes the problem then. Thank you for the testing!