Closed t-b closed 5 years ago
Conversion script:
$cat run.sh
#!/bin/bash
rm -f /tmp/git
touch /tmp/git
# generate a list of files we want to remove
git log --numstat --name-only --format="" --all --ignore-submodules | grep -v -P ".(ipf|ipfT[0-9]|ibw|ihf|awk|Doxyfile|md|rst|gitignore|hgflow|txt|pxp|sh|nsi|dot|svg|itx|nwb|gitmodules|gitattributes|bat|lnk|py|template|cnf|mailmap|ift|vbs|placeholder|pl|otf|tex|sty|css|clang-format)$" | grep -v -P "(LICENSE|Dockerfile|mies.ico|tools/nircmd/.*|libzmq-.*.dll|ZeroMQ.*\.xop|Arduino-libraries-and-sequencer.zip|MultiClamp700xCommander.*\.xop|itcXOP2.*\.xop|ITCMM.*\.dll|MIESUtils.*\.xop|vc_redist.*\.exe|Packages/ZeroMQ.*)$" | sed -e 's;^Packages/ITC.*;Packages/ITC;' | sed -e 's;^tools/installer/nsis.*;tools/installer/nsis;' | grep -v "^XOP-tango" | sort | uniq > /tmp/files-to-remove.txt
# filter are executed in the predefined and here given order:
#
# index-filter takes care of removing the unwanted files
# msg-filter adds the Original-CommitID and fills /tmp/git for the old sha->new sha translation with old sha
# commit-filter fills /tmp/git for the old sha->new sha translation with new sha
git filter-branch \
--index-filter 'git rm --quiet -r --cached --ignore-unmatch $(cat /tmp/files-to-remove.txt)' \
--msg-filter 'sed -f /tmp/git; \
echo "\nOriginal-CommitID: ${GIT_COMMIT}";' \
--commit-filter 'OLD_COMMIT_SHORT=$(echo ${GIT_COMMIT} | cut -c 1-8); \
echo -n "s/\(${OLD_COMMIT_SHORT}\|${GIT_COMMIT}|${GIT_COMMIT}\)/" >>/tmp/git; \
NEW=`git_commit_non_empty_tree "$@"`; \
NEW_COMMIT_SHORT=$(echo ${NEW} | cut -c 1-8); \
echo "${NEW_COMMIT_SHORT}/g" >> /tmp/git; echo ${NEW}' \
--tag-name-filter cat \
-- --all
# do the old sha->new sha transition on our code base
# could be done as tree filter as well but would make everything *much* slower
sed -i -f /tmp/git Packages/MIES/*.ipf Packages/Testing-MIES/*.ipf
rm -f /tmp/msg
echo -e "Public-Repository: Replace old SHAs" >> /tmp/msg
git commit --allow-empty -F /tmp/msg Packages/MIES/*.ipf Packages/Testing-MIES
rm -f /tmp/msg
# add this script as commit message
echo -e "Public-Repository: Conversion was done with the following script\n\n" >> /tmp/msg
cat ${0} >> /tmp/msg
echo -e "\n\nand the files to be removed where\n" >> /tmp/msg
cat /tmp/files-to-remove.txt >> /tmp/msg
git commit --allow-empty -F /tmp/msg
We can not push the cleaned up repo as it is too large
$git push --force origin master
Objekte aufzählen: 31109, Fertig.
Zähle Objekte: 100% (31109/31109), Fertig.
Delta-Kompression verwendet bis zu 12 Threads.
Komprimiere Objekte: 100% (7798/7798), Fertig.
Schreibe Objekte: 100% (31109/31109), 111.93 MiB | 2.85 MiB/s, Fertig.
Gesamt 31109 (Delta 24343), Wiederverwendet 29814 (Delta 23253)
remote: Resolving deltas: 100% (24343/24343), done.
remote: warning: File tools/unit-testing/RunAllTests.pxp is 71.43 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File tools/unit-testing/RunAllTests.pxp is 99.04 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File tools/unit-testing/RunAllTests.pxp is 65.35 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File tools/unit-testing/RunAllTests.pxp is 56.29 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File tools/unit-testing/HardwareTests.pxp is 88.86 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File tools/unit-testing/HardwareTests.pxp is 85.59 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: error: GH001: Large files detected. You may want to try Git Large File Storage - https://git-lfs.github.com.
remote: error: Trace: 672f7bb55fee4e398c7fd2059473ac6c
remote: error: See http://git.io/iEPt8g for more information.
remote: error: File tools/unit-testing/RunAllTests.pxp is 117.58 MB; this exceeds GitHub's file size limit of 100.00 MB
To github.com:AllenInstitute/MIES.git
! [remote rejected] master -> master (pre-receive hook declined)
error: Fehler beim Versenden einiger Referenzen nach 'git@github.com:AllenInstitute/MIES.git'