svn-all-fast-export / svn2git

:octopus: A fast-import based converter for an svn repo to git repos
GNU General Public License v3.0
262 stars 100 forks source link

Getting slower and slower on merged revisions and recorded revisions, non-linear runtime ? #144

Closed achimmihca closed 2 years ago

achimmihca commented 2 years ago

I am trying to migrate a repo with over 300,000 commits.

In the beginning, the tool processes about 500 commits per second. For the first 50,000 commits, it only takes 1-2 minutes. But this slows down dramatically. At commit 100,000 some commits need several seconds for the migration. And after 16 hours the tool only reached commit 150,000.

Looking at the log, this seems to be an issue with merged revisions and recorded revisions (svn option --record-only).

These commits take a lot of time.

Other non-merge commits behave normally, much much faster.

Is there anything I can do to improve performance of merge commits?

achimmihca commented 2 years ago

Not all of our merge commits are affected.

I think it is related to changes of svn folder properties. This is what changes in a record-only merge. And there is always a changed folder in the other slow merge commits.

Is there an option to ignore changes to svn folder properties?

achimmihca commented 2 years ago

I found the problem: I was using the option --svn-ignore to generate a .gitignore file from SVN properties. Without this option every revision is converted fast, including all merges and recorded revisions.

I guess the options reads the folder properties from SVN and therefor, also merges the folder properties, which is taking longer and longer in my case.

Please add a warning about this performance impact to the option's desciption. Otherwise great and very fast tool, thanks a lot!

Sorry for the inconvenience.