rtyley / bfg-repo-cleaner

Removes large or troublesome blobs like git-filter-branch does, but faster. And written in Scala
https://rtyley.github.io/bfg-repo-cleaner/
GNU General Public License v3.0
10.95k stars 540 forks source link

Getting InvocationTargetException when converting files separately #314

Closed Sparcks closed 5 years ago

Sparcks commented 5 years ago

Hi

I tried the BFG repo cleaner and first it looked good, but I did not see any .gitattribute anywhere. I used following command first: java -jar bfg.jar --convert-to-git-lfs "*{_database,_generic,manifest,.md5}" --no-blob-protection myrepo.git

Here you can see I have files with no extension, because they are just created like this. But it seemed to work, untill I noticed there was no .gitattribute file. Don't think this should be the case, but maybe it is because of having files with no extension?

Anyway, afterwards I tried them separately (started from scratch of course), but that gave me an exception:

java.lang.reflect.InvocationTargetException

I used these 2 commands, but it happend on the second command: java -jar bfg.jar --convert-to-git-lfs "*_database" --no-blob-protection myrepo.git java -jar bfg.jar --convert-to-git-lfs "*_generic" --no-blob-protection myrepo.git

Full output:

Using repo : C:\Develop\Git\myrepo.git This repo has been processed by The BFG before! Will prune repo before proceeding - to avoid unnecessary cleaning work on unused objects... Completed prune of old objects - will now proceed with the main job! Found 0 objects to protect java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at use.newer.java.Util.invokeMainMethod(Util.java:89) at use.newer.java.Util.executeMainProgram(Util.java:28) at use.newer.java.Util.checkJavaVersionAndExecuteMainProgram(Util.java:18) at use.newer.java.Version8.main(Version8.java:5) Caused by: org.eclipse.jgit.errors.MissingObjectException: Missing unknown fd6d254190d0bff22d3283bb5dc74dfa6111e4c5 at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:158) at org.eclipse.jgit.lib.ObjectReader.open(ObjectReader.java:227) at org.eclipse.jgit.revwalk.RevWalk.parseAny(RevWalk.java:859) at com.madgag.git.package$RichObjectId.asRevObject(package.scala:225) at com.madgag.git.bfg.cleaner.RepoRewriter$.$anonfun$rewrite$2(RepoRewriter.scala:84) at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:234) at scala.collection.Iterator.foreach(Iterator.scala:929) at scala.collection.Iterator.foreach$(Iterator.scala:929) at scala.collection.AbstractIterator.foreach(Iterator.scala:1417) at scala.collection.IterableLike.foreach(IterableLike.scala:71) at scala.collection.IterableLike.foreach$(IterableLike.scala:70) at scala.collection.AbstractIterable.foreach(Iterable.scala:54) at scala.collection.TraversableLike.map(TraversableLike.scala:234) at scala.collection.TraversableLike.map$(TraversableLike.scala:227) at scala.collection.AbstractTraversable.map(Traversable.scala:104) at com.madgag.git.bfg.cleaner.RepoRewriter$.createRevWalk$1(RepoRewriter.scala:84) at com.madgag.git.bfg.cleaner.RepoRewriter$.rewrite(RepoRewriter.scala:90) at com.madgag.git.bfg.cli.Main$.$anonfun$new$1(Main.scala:60) at com.madgag.git.bfg.cli.Main$.$anonfun$new$1$adapted(Main.scala:35) at scala.Option.map(Option.scala:146) at com.madgag.git.bfg.cli.Main$.delayedEndpoint$com$madgag$git$bfg$cli$Main$1(Main.scala:35) at com.madgag.git.bfg.cli.Main$delayedInit$body.apply(Main.scala:28) at scala.Function0.apply$mcV$sp(Function0.scala:34) at scala.Function0.apply$mcV$sp$(Function0.scala:34) at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12) at scala.App.$anonfun$main$1$adapted(App.scala:76) at scala.collection.immutable.List.foreach(List.scala:389) at scala.App.main(App.scala:76) at scala.App.main$(App.scala:74) at com.madgag.git.bfg.cli.Main$.main(Main.scala:28) at com.madgag.git.bfg.cli.Main.main(Main.scala) ... 8 more

I used the latest bfg.jar file (1.13.0) and I ran the commands in Git Bash on Windows 10. Does someone see if I'm doing something wrong here?

Thanks in advance.

javabrett commented 5 years ago

Try running the documented git gc command between runs. Also note that bfg is not necessarily the current recommended tool for lfs conversions.

Sparcks commented 5 years ago

Try running the documented git gc command between runs. Also note that bfg is not necessarily the current recommended tool for lfs conversions.

Hi @javabrett

I did run the recommended commandline: git gc --prune=now --aggressive But this takes a REALLY long time for the repository. It's quite big already with the history and all. So doing it after every command is too time consuming.

Or do you mean just running git gc only, without the extra options? Can you think of other tools that could help with conversions to lfs?

Thanks!

javabrett commented 5 years ago

BFG can run into data/recursion issues if it is run multiple times without an interceding gc. There are some unmerged PRs which allow certain types of multiple operations in one run.

Other than that, I am afraid you are stuck running the full gc each time.

Git LFS has its own migrate command nowadays - you might have success with that.

Sparcks commented 5 years ago

BFG can run into data/recursion issues if it is run multiple times without an interceding gc. There are some unmerged PRs which allow certain types of multiple operations in one run.

Other than that, I am afraid you are stuck running the full gc each time.

Git LFS has its own migrate command nowadays - you might have success with that.

Okay, I'll have a look. Thanks.