search

nirvdrum / svn2git

Ruby tool for importing existing svn projects into git.
MIT License
2.11k stars 443 forks source link

Performance for relatively large repository #125

Closed obones closed 11 years ago

obones commented 11 years ago

Hello all,

I'm currently doing some tests for importing the JVCL project SVN content and as recommended by GitHub, I'm using svn2git There are about 13500 revisions to import and after 24 hours running at 100% CPU, only 3000 have been imported. Is this kind of speed something that is expected? What can I do to speed up the process?

Thanks in advance for your answers Regards

jasonwbarnett commented 11 years ago

I can only speak from personal experience, but that kind of sounds normal. I used a 24 core machine with 96GB of memory and it took a while to convert a 26000 revision repository. I can't say how long a while is, but I definitely left the process running overnight and then some.

kayhadrin commented 11 years ago

I concur. I tried to convert ~110000 svn revisions to git, but could only get 80000 processed after about 4 weeks. The conversion time needed for each additional commit was growing almost exponentially. I eventually had to give up and converted the last 8000 commits from svn instead.

On 31 May 2013 23:48, Jason Barnett notifications@github.com wrote:

I can only speak from personal experience, but that kind of sounds normal. I used a 24 core machine with 96GB of memory and it took a while to convert a 26000 revision repository.

— Reply to this email directly or view it on GitHubhttps://github.com/nirvdrum/svn2git/issues/125#issuecomment-18745713 .

obones commented 11 years ago

Thanks for your replies, it confirms my observations. What's misleading is that it does a complete import from revision 1 for all branches so looking at the revision number in verbose mode is not very safe. But moving the import process to a 4 core machine with 2GB of RAM allowed to finish it in just above 48 hours because we had 47 branches/tags to import.

I'm closing the issue now, but it would be nice if the documentation would be clearer on the speed of the process.

jasonwbarnett commented 10 years ago

I would like to follow up to this issue in case anyone else stumbles upon it.

I just recently attempted to convert the exact same repository which now has a lot more revisions (78942 total, but I started at 42738 which is a total of 36204 converted) and it was much faster. I used svn2git v2.3.2 + Git v2.1.0. I was using a GCE VM (n1-standard-2 (2 vCPU, 7.5 GB memory)) + 100GB SSD persistent disk.

I am guessing the improved performance is largely due to Git v2.1.0 improvements and the SSD. It took about 12 hours to convert the SVN repo.