hanw / git-repo

Automatically exported from code.google.com/p/git-repo
Apache License 2.0
0 stars 0 forks source link

Parallell run of git gc hangs #126

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago
Affected Version:
repo version v1.11.1
       (from https://android.googlesource.com/tools/repo)
repo launcher version 1.18
       (from /home/bjorn/sandbox/lt26/.repo/repo/repo)

Environment:
git version 1.7.5.4
Python 2.7.2+ (default, Jul 20 2012, 22:15:08) 
[GCC 4.6.1]

What steps will reproduce the problem?
repo sync -j2

What is the expected output? What do you see instead?
During the git gc phase, after fetching all projects, repo sometimes stalls if 
being run with jobs >= 2. Higher numer of jobs increases the probability of 
this issue to occur, with -j4 it rarely every finishes.

Please provide any additional information below.
Repo hangs while joining the threads in _GCProjects(), because not all threads 
are done (all git processes are finished according to ps).

The issue was introduced by 188572170e8cdf28df55a5ca90ed70d14b72b804.

Original issue reported on code.google.com by bjorn.an...@sonymobile.com on 25 Nov 2012 at 10:01

GoogleCodeExporter commented 9 years ago
After some more debugging I've concluded that this is caused by a deadlock in 
python when fork is used in a threaded program, as reported in 
http://bugs.python.org/issue13817. A fix for this is integrated in 2.7.3.

So, please close this issue.

Original comment by bjorn.an...@sonymobile.com on 29 Nov 2012 at 3:21

GoogleCodeExporter commented 9 years ago
For the record:

The problem is that thread local storage is emulated in some versions of python 
and every access is guarded by a global mutex, the thread local storage is 
re-initialized right after a fork() syscall, which is done with the mutex held. 
So if any other thread holds the global mutex over the fork() call a deadlock 
will occur.

This is fixed by cleaning up the order that things are re-initialized after the 
fork() call.

So any python application calling fork while having other threads running is 
likely to hit this issue sooner or later.

For the python 2.7 series, that my machine runs, the issue is corrected in 
2.7.3.

Original comment by bjorn.an...@sonymobile.com on 30 Nov 2012 at 1:38