mspncp
commented
4 years ago The script to reproduce the issue
Place the show-issue script into an empty directory and make it executable, then run it. The script can be executed several times in which case it will reuse the existing SVN repository, unless you specify the -r (recreate) option.
show-issue
#!/bin/bash
set -o errexit
# use -r option to recreate the issue
if [ "$1" == "-r" ]; then
rm -rf issue
fi
mkdir -p issue
cd issue
svn_url="file://$(readlink -f example.svn)"
if [ ! -e example.svn ]; then
svnadmin create example.svn
# COMMIT 1 - Create ttb structure
svn mkdir $svn_url/trunk $svn_url/tags $svn_url/branches -m "Create ttb structure"
svn co $svn_url/trunk example
cd example
mkdir src other
for f in foo bar baz; do
echo $f > $f.txt
echo $f > src/$f.c
echo $f > other/$f.py
done
# COMMIT 2 - Add my files
svn add *.txt
svn add src
svn ci -m "Add my files"
# COMMIT 3 - Add other files
svn add other
svn ci -m "Add other files"
# COMMIT 4 - Create feature branch
svn cp $svn_url/trunk $svn_url/branches/feature -m "Create feature branch"
# COMMIT 5 - Add some stuff to my files
echo "some stuff" >> bar.txt
echo "some stuff" >> src/bar.c
svn ci -m "Add some stuff to my files"
# COMMIT 7 - Add some stuff to other files
echo "some stuff" >> other/bar.py
svn ci -m "Add some stuff to other files"
svn switch ^/branches/feature
# COMMIT 8 - Add more stuff to my files
echo "more stuff" >> bar.txt
echo "more stuff" >> src/bar.c
svn ci -m "Add more stuff to my files"
# COMMIT 10 - Add some stuff to other files
echo "more stuff" >> other/bar.py
svn ci -m "Add more stuff to other files"
cd ..
fi
cat > broken.rules <<EOF
create repository broken/main.git
end repository
create repository broken/other.git
end repository
match /trunk/other/
repository broken/other.git
branch master
end match
match /trunk/
repository broken/main.git
branch master
end match
match /branches/([^/]+)/other/
repository broken/other.git
branch \1
end match
match /branches/([^/]+)/
repository broken/main.git
branch \1
end match
EOF
cat > main.rules <<EOF
create repository fixed/main.git
end repository
match /trunk/other/
action ignore
end match
match /trunk/
repository fixed/main.git
branch master
end match
match /branches/([^/]+)/other/
action ignore
end match
match /branches/([^/]+)/
repository fixed/main.git
branch \1
end match
EOF
cat > other.rules <<EOF
create repository fixed/other.git
end repository
match /trunk/other/
repository fixed/other.git
branch master
end match
match /branches/([^/]+)/$
action recurse
end match
match /branches/([^/]+)/other/
repository fixed/other.git
branch \1
end match
match /
action ignore
end match
EOF
mkdir -p broken
(
set -o xtrace
svn-all-fast-export --debug-rules --add-metadata --rules broken.rules example.svn
) |& tee broken.log
mkdir -p fixed
(
set -o xtrace
svn-all-fast-export --debug-rules --add-metadata --rules main.rules,other.rules example.svn
) |& tee fixed.log
echo
echo "##"
echo "## example.svn"
echo "##"
svn log -v $svn_url > example.log
for solution in broken fixed; do
(
for repo in main other; do
r=$solution/$repo.git
echo
echo "##"
echo "## $r"
echo "##"
git -C $r log --all --graph --stat master feature
for b in master feature; do
echo
echo "tree of $b branch:"
git -C $r ls-tree -r $b
done
done
) |& tee -a $solution.log
done
Vampire
I am currently migrating a large SVN repository to Git which has a monolithic TTB structure (meaning: no subprojects, all branches are copies of the entire
^/trunk). During the conversion, most of the repository should go into a single large Git repository, except for one directory, which should move to a separate repository. I had some problems doing the conversion which I would like to describe first before asking some questions. To demonstrate my problem, I prepared a simplified toy example.Attached below the issue you will find a bash script (
show-issue) to reproduce the toy example from scratch.The SVN repository
The SVN repository (
example.svn) has two branches, the trunk (^/trunk) and a feature branch (^/branches/feature). Both branches have the same directory structure:As one can see from the following history, the feature branch was branched off the trunk in r4 and both branches received additional commits.
First migration attempt (broken)
All files from the
example.svnrepository should go into a single large Git repositorymain.git, except for the contents of theotherdirectory, which go toother.git. This here was my first attempt:broken.rules
The Outcome
broken.log
The content of the
main.gitrepository looks ok:But the feature branch of the
other.gitrepository is 'detached' from themasterbranch and its tree is incomplete, i.e., it contains only the single file (bar.py) which was modified after branching (in r8).The Problem
I learned from UsingSvn2Git that the rule
won't be matched in r4 (
A /branches/feature (from /trunk:3)), unless I define a recurse actionHowever, I also need an export action for the main repository
and unfortunately
svn-all-fast-exportdoes not support an 'export-and-recurse' action.Second migration attempt
The only solution I could come up with to fix that conflict was to create two separate rule files:
main.rules
other.rules
MultiRules!!
Initially, the downside of this approach was that I had to call
svn-all-fast-exporttwice, which doubled the conversion time. But then I discovered (from the source code) a handy feature, namely that you can specify a comma separated list of rules on the command line.The two rule files are not merged, but instead executed independently in parallel. This feature was added in commit a741bdb1913c by @tnyblom. This is an excellent feature IMO and unfortunately poorly documented.
The Outcome
This time, the result looks good:
fixed.log
Questions
Having to maintain two different rule files makes things a little bit complicated (in particular, since my real repository is not as simple as my example) and I was asking myself whether there was a different way to solve my problem using a single rules file, which I might have missed?
If not, would it be a great effort @tnyblom to add a new 'export-and-recurse' action and would this make it possible to have all rules in a single file (as in the broken attempt) using something like the following?