search

svn-all-fast-export / svn2git

:octopus: A fast-import based converter for an svn repo to git repos
GNU General Public License v3.0
262 stars 100 forks source link

Segmentation fault processing svn repository #26

Open eeijlar opened 7 years ago

eeijlar commented 7 years ago

I get a segmentation fault while processing a SVN repository. There is no information other than:

fatal: EOF in data (285 bytes remaining)

It creates a dump report but I don't know what the contents means. It prints '+' to the screen over and over and it never appears to move past that point. It's like it has encountered a circular dependency in the svn repository.

Here's the back trace from the core dump:

[New LWP 26037]
[New LWP 26038]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `svn2git/svn-all-fast-export --identity-map ../tools/authors/authors.txt --rules'.
Program terminated with signal 11, Segmentation fault.
#0  0x00007faaad763de1 in QByteArray::append (this=this@entry=0x189ef78, ba=...) at tools/qbytearray.cpp:1616
1616            d->data[d->size] = '\0';
(gdb) bt
#0  0x00007faaad763de1 in QByteArray::append (this=this@entry=0x189ef78, ba=...) at tools/qbytearray.cpp:1616
#1  0x0000000000418c61 in append (s=..., this=0x189ef78) at /usr/include/QtCore/qstring.h:997
#2  FastImportRepository::Transaction::addFile (this=0x189ef30, path=..., mode=33261, length=236) at repository.cpp:899
#3  0x000000000042259e in dumpBlob (txn=txn@entry=0x189ef30, fs_root=fs_root@entry=0xd3f6f0, pathname=<optimized out>, finalPathName=..., pool=pool@entry=0x7faa99e92038) at svn.cpp:292
#4  0x0000000000423fff in recursiveDumpDir (txn=txn@entry=0x189ef30, fs_root=fs_root@entry=0xd3f6f0, pathname=..., finalPathName=..., pool=pool@entry=0xdd2218) at svn.cpp:340
#5  0x0000000000423f53 in recursiveDumpDir (txn=txn@entry=0x189ef30, fs_root=fs_root@entry=0xd3f6f0, pathname=..., finalPathName=..., pool=pool@entry=0xda01d8) at svn.cpp:335
#6  0x0000000000423f53 in recursiveDumpDir (txn=txn@entry=0x189ef30, fs_root=fs_root@entry=0xd3f6f0, pathname=..., finalPathName=..., pool=pool@entry=0xd8f1c8) at svn.cpp:335
#7  0x0000000000423f53 in recursiveDumpDir (txn=txn@entry=0x189ef30, fs_root=fs_root@entry=0xd3f6f0, pathname=..., finalPathName=..., pool=pool@entry=0xdc6208) at svn.cpp:335
#8  0x0000000000423f53 in recursiveDumpDir (txn=txn@entry=0x189ef30, fs_root=fs_root@entry=0xd3f6f0, pathname=..., finalPathName=..., pool=pool@entry=0xd6b188) at svn.cpp:335
#9  0x0000000000423f53 in recursiveDumpDir (txn=txn@entry=0x189ef30, fs_root=fs_root@entry=0xd3f6f0, pathname=..., finalPathName=..., pool=pool@entry=0xdbb1f8) at svn.cpp:335
#10 0x0000000000423f53 in recursiveDumpDir (txn=txn@entry=0x189ef30, fs_root=fs_root@entry=0xd3f6f0, pathname=..., finalPathName=..., pool=pool@entry=0xdb11e8) at svn.cpp:335
#11 0x0000000000423f53 in recursiveDumpDir (txn=txn@entry=0x189ef30, fs_root=fs_root@entry=0xd3f6f0, pathname=..., finalPathName=..., pool=pool@entry=0xd861b8) at svn.cpp:335
#12 0x0000000000423f53 in recursiveDumpDir (txn=txn@entry=0x189ef30, fs_root=0xd3f6f0, pathname=..., finalPathName=..., pool=pool@entry=0xcf81f8) at svn.cpp:335
#13 0x0000000000426e07 in SvnRevision::exportInternal (this=this@entry=0x7fff44b63ca0, key=key@entry=0x189d8d8 "/FusionWorks/branches/FW_5_5_0_226", change=change@entry=0xcf9780,
    path_from=<optimized out>, path_from@entry=0xd4ee87 "/branches", rev_from=rev_from@entry=149644, current=..., rule=..., matchRules=...) at svn.cpp:914
#14 0x000000000042b56b in SvnRevision::exportDispatch (this=this@entry=0x7fff44b63ca0, key=key@entry=0x189d8d8 "/branches/FW_5_5_0_226", change=change@entry=0xcf9780,
    path_from=0xd4ee87 "/FusionWorks/branches", rev_from=149644, changes=changes@entry=0xcf96b0, current=..., rule=..., matchRules=..., pool=pool@entry=0xd4ee08) at svn.cpp:716
#15 0x000000000042cff0 in SvnRevision::exportEntry (this=this@entry=0x7fff44b63ca0, key=0x189d8d8 "/FusionWorks/branches/FW_5_5_0_226", change=0xcf9780, changes=0xcf96b0) at svn.cpp:666
#16 0x000000000042f091 in SvnRevision::prepareTransactions (this=this@entry=0x7fff44b63ca0) at svn.cpp:536
#17 0x000000000042f660 in SvnPrivate::exportRevision (this=<optimized out>, revnum=revnum@entry=149645) at svn.cpp:444
#18 0x000000000042f7ec in Svn::exportRevision (this=this@entry=0x7fff44b63e40, revnum=revnum@entry=149645) at svn.cpp:154
#19 0x0000000000407754 in main (argc=6, argv=<optimized out>) at main.cpp:276
(gdb)
eeijlar commented 7 years ago

Luckily the branch which caused the problem is in the stack trace, so I just did:

match /branches/broken_branch_name
end match

This allowed svn2git to skip the branch and continue on with the conversion. The errant commit seems to have propagated right through the history though. It takes about 2 hours for the process to core and before you can find the branch.

See also: http://stackoverflow.com/questions/42828855/error-importing-svn-report-to-git-using-svn2git

It would be nice if it was able to log the branch that is causing trouble. It looks like it only logs the branch after it processes it.

eeijlar commented 7 years ago

In some cases you could be waiting for 6 hours for it to core dump, a better alternative is to do : 

pstack <pid>

This should give you the branch it's currently processing, which you can then add to your rule set for skipping.

eeijlar commented 7 years ago

I think this maybe caused by a circular dependency within the svn commit history. For example if you have a project with /MyProject/branches/ and you create a branch off it /MyProject/branches/my_dev_branch/ Then if you copy the contents of /MyProject/branches/ into /MyProject/branches/my_dev_branch/

Vampire commented 4 years ago

We have now a test harness in place. Could you maybe write a little test case that reproduces the problem?