search

rtyley / bfg-repo-cleaner

Removes large or troublesome blobs like git-filter-branch does, but faster. And written in Scala
https://rtyley.github.io/bfg-repo-cleaner/
GNU General Public License v3.0
10.86k stars 535 forks source link

Getting java.lang.reflect.InvocationTargetException #361

Open gyosko opened 4 years ago

gyosko commented 4 years ago

When I run

bfg --strip-blobs-bigger-than 100M prod.git

This is the complete output I get:

Scanning packfile for large blobs: 63086
Scanning packfile for large blobs completed in 796 ms.
Found 2 blob ids for large blobs - biggest=895599494 smallest=138902312
Total size (unpacked)=1034501806
java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:567)
    at use.newer.java.Util.invokeMainMethod(Util.java:89)
    at use.newer.java.Util.executeMainProgram(Util.java:28)
    at use.newer.java.Util.checkJavaVersionAndExecuteMainProgram(Util.java:18)
    at use.newer.java.Version8.main(Version8.java:5)
Caused by: org.eclipse.jgit.errors.LargeObjectException: 531d55ace05be4b5abefd5f955628c77e5c9d433 exceeds size limit
    at org.eclipse.jgit.internal.storage.file.LargePackedWholeObject.getCachedBytes(LargePackedWholeObject.java:98)
    at org.eclipse.jgit.treewalk.CanonicalTreeParser.reset(CanonicalTreeParser.java:214)
    at org.eclipse.jgit.treewalk.CanonicalTreeParser.createSubtreeIterator0(CanonicalTreeParser.java:248)
    at org.eclipse.jgit.treewalk.CanonicalTreeParser.createSubtreeIterator(CanonicalTreeParser.java:226)
    at org.eclipse.jgit.treewalk.CanonicalTreeParser.createSubtreeIterator(CanonicalTreeParser.java:70)
    at org.eclipse.jgit.treewalk.TreeWalk.enterSubtree(TreeWalk.java:1210)
    at org.eclipse.jgit.treewalk.TreeWalk.next(TreeWalk.java:828)
    at com.madgag.git.package$RichTreeWalk$$anon$1.next(package.scala:135)
    at scala.collection.Iterator.foreach(Iterator.scala:929)
    at scala.collection.Iterator.foreach$(Iterator.scala:929)
    at com.madgag.git.package$RichTreeWalk$$anon$1.foreach(package.scala:128)
    at scala.collection.generic.Growable.$plus$plus$eq(Growable.scala:59)
    at scala.collection.generic.Growable.$plus$plus$eq$(Growable.scala:50)
    at scala.collection.mutable.SetBuilder.$plus$plus$eq(SetBuilder.scala:20)
    at scala.collection.TraversableOnce.to(TraversableOnce.scala:310)
    at scala.collection.TraversableOnce.to$(TraversableOnce.scala:308)
    at com.madgag.git.package$RichTreeWalk$$anon$1.to(package.scala:128)
    at scala.collection.TraversableOnce.toSet(TraversableOnce.scala:304)
    at scala.collection.TraversableOnce.toSet$(TraversableOnce.scala:304)
    at com.madgag.git.package$RichTreeWalk$$anon$1.toSet(package.scala:128)
    at com.madgag.git.package$.allBlobsUnder(package.scala:268)
    at com.madgag.git.bfg.cleaner.protection.ProtectedObjectCensus$.$anonfun$apply$5(ProtectedObjectCensus.scala:83)
    at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:241)
    at scala.collection.Iterator.foreach(Iterator.scala:929)
    at scala.collection.Iterator.foreach$(Iterator.scala:929)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1417)
    at scala.collection.MapLike$DefaultKeySet.foreach(MapLike.scala:178)
    at scala.collection.TraversableLike.flatMap(TraversableLike.scala:241)
    at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:238)
    at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
    at com.madgag.git.bfg.cleaner.protection.ProtectedObjectCensus$.apply(ProtectedObjectCensus.scala:83)
    at com.madgag.git.bfg.cli.CLIConfig.objectProtection$lzycompute(CLIConfig.scala:145)
    at com.madgag.git.bfg.cli.CLIConfig.objectProtection(CLIConfig.scala:145)
    at com.madgag.git.bfg.cli.Main$.$anonfun$new$1(Main.scala:58)
    at com.madgag.git.bfg.cli.Main$.$anonfun$new$1$adapted(Main.scala:35)
    at scala.Option.map(Option.scala:146)
    at com.madgag.git.bfg.cli.Main$.delayedEndpoint$com$madgag$git$bfg$cli$Main$1(Main.scala:35)
    at com.madgag.git.bfg.cli.Main$delayedInit$body.apply(Main.scala:28)
    at scala.Function0.apply$mcV$sp(Function0.scala:34)
    at scala.Function0.apply$mcV$sp$(Function0.scala:34)
    at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
    at scala.App.$anonfun$main$1$adapted(App.scala:76)
    at scala.collection.immutable.List.foreach(List.scala:389)
    at scala.App.main(App.scala:76)
    at scala.App.main$(App.scala:74)
    at com.madgag.git.bfg.cli.Main$.main(Main.scala:28)
    at com.madgag.git.bfg.cli.Main.main(Main.scala)
    ... 8 more

#314 seems related, but running git gc didn't solve my issue.

I'm on MacOS and I installed java using brew.

java -version
openjdk version "13.0.2" 2020-01-14
OpenJDK Runtime Environment (build 13.0.2+8)
OpenJDK 64-Bit Server VM (build 13.0.2+8, mixed mode, sharing)
rugk commented 3 years ago

This happened to me when trying to use this on a network drive on Windows, maybe you do have similar problems.

onescore-agray commented 1 year ago

@gyosko did you ever find a solution? I'm getting the same error.

gyosko commented 1 year ago

@onescore-agray no, but I didn't try anymore

ElectricRCAircraftGuy commented 1 year ago

I'm getting the same error running bfg 1.14.0 on Ubuntu 20.04:

My java info:

$ java --version
openjdk 11.0.15 2022-04-19
OpenJDK Runtime Environment (build 11.0.15+10-Ubuntu-0ubuntu0.20.04.1)
OpenJDK 64-Bit Server VM (build 11.0.15+10-Ubuntu-0ubuntu0.20.04.1, mixed mode, sharing)

Full bfg command and output:

$ bfg --delete-files "Gmail - Project [handed in 20210722-0025hrs! See email p9 & 11-12].pdf"

Using repo : /home/gabriel/GS/dev/my_repo/.git

Found 72 objects to protect
Found 4 commit-pointing refs : HEAD, refs/heads/main, refs/remotes/origin/HEAD, refs/remotes/origin/main
Found 2 tag-pointing refs : refs/tags/zipline_takehome_test_sent_updated_readme, refs/tags/zipline_takehome_test_submitted

Protected commits
-----------------

These are your protected commits, and so their contents will NOT be altered:

java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at use.newer.java.Util.invokeMainMethod(Util.java:89)
    at use.newer.java.Util.executeMainProgram(Util.java:28)
    at use.newer.java.Util.checkJavaVersionAndExecuteMainProgram(Util.java:18)
    at use.newer.java.Version8.main(Version8.java:5)
Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.util.regex.PatternSyntaxException: Invalid range near index 44
Gmail - Project [handed in 20210722-0025hrs! See email p9 & 11-12].pdf
                                            ^
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2051)
    at com.google.common.cache.LocalCache.get(LocalCache.java:3951)
    at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
    at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4935)
    at com.madgag.git.bfg.MemoUtil$$anon$2.apply(memo.scala:60)
    at com.madgag.git.bfg.GitUtil$$anon$1.apply(GitUtil.scala:71)
    at com.madgag.git.bfg.CleaningMapper.replacement(GitUtil.scala:46)
    at com.madgag.git.bfg.CleaningMapper.replacement$(GitUtil.scala:45)
    at com.madgag.git.bfg.GitUtil$$anon$1.replacement(GitUtil.scala:70)
    at com.madgag.git.bfg.cleaner.protection.ProtectedObjectDirtReport$.$anonfun$reportsFor$3(ProtectedObjectDirtReport.scala:45)
    at scala.util.Either.fold(Either.scala:189)
    at com.madgag.git.bfg.cleaner.protection.ProtectedObjectDirtReport$.$anonfun$reportsFor$1(ProtectedObjectDirtReport.scala:45)
    at scala.collection.Iterator$$anon$9.next(Iterator.scala:575)
    at scala.collection.mutable.Growable.addAll(Growable.scala:65)
    at scala.collection.mutable.Growable.addAll$(Growable.scala:60)
    at scala.collection.immutable.SetBuilderImpl.addAll(Set.scala:396)
    at scala.collection.immutable.Set$.from(Set.scala:353)
    at scala.collection.immutable.Set$.from(Set.scala:93)
    at scala.collection.IterableOps.map(Iterable.scala:671)
    at scala.collection.IterableOps.map$(Iterable.scala:671)
    at scala.collection.AbstractIterable.map(Iterable.scala:919)
    at com.madgag.git.bfg.cleaner.protection.ProtectedObjectDirtReport$.reportsFor(ProtectedObjectDirtReport.scala:43)
    at com.madgag.git.bfg.cleaner.CLIReporter.reportProtectedCommitsAndTheirDirt(Reporter.scala:133)
    at com.madgag.git.bfg.cleaner.CLIReporter.reportObjectProtection(Reporter.scala:106)
    at com.madgag.git.bfg.cleaner.RepoRewriter$.rewrite(RepoRewriter.scala:96)
    at com.madgag.git.bfg.cli.Main$.$anonfun$new$1(Main.scala:59)
    at com.madgag.git.bfg.cli.Main$.$anonfun$new$1$adapted(Main.scala:34)
    at scala.Option.map(Option.scala:242)
    at com.madgag.git.bfg.cli.Main$.delayedEndpoint$com$madgag$git$bfg$cli$Main$1(Main.scala:34)
    at com.madgag.git.bfg.cli.Main$delayedInit$body.apply(Main.scala:27)
    at scala.Function0.apply$mcV$sp(Function0.scala:39)
    at scala.Function0.apply$mcV$sp$(Function0.scala:39)
    at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
    at scala.App.$anonfun$main$1(App.scala:76)
    at scala.App.$anonfun$main$1$adapted(App.scala:76)
    at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:563)
    at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:561)
    at scala.collection.AbstractIterable.foreach(Iterable.scala:919)
    at scala.App.main(App.scala:76)
    at scala.App.main$(App.scala:74)
    at com.madgag.git.bfg.cli.Main$.main(Main.scala:27)
    at com.madgag.git.bfg.cli.Main.main(Main.scala)
    ... 8 more
Caused by: java.util.regex.PatternSyntaxException: Invalid range near index 44
Gmail - Project [handed in 20210722-0025hrs! See email p9 & 11-12].pdf
                                            ^
    at com.madgag.globs.openjdk.Globs.toRegexPattern(Globs.java:135)
    at com.madgag.globs.openjdk.Globs.toUnixRegexPattern(Globs.java:211)
    at com.madgag.textmatching.Glob$.regexFor(TextMatcherType.scala:64)
    at com.madgag.textmatching.TextMatcher.r$lzycompute(TextMatcherType.scala:40)
    at com.madgag.textmatching.TextMatcher.r(TextMatcherType.scala:40)
    at com.madgag.textmatching.TextMatcher.apply(TextMatcherType.scala:42)
    at com.madgag.git.bfg.cleaner.FileDeleter.$anonfun$apply$1(treeblobs.scala:30)
    at com.madgag.git.bfg.cleaner.FileDeleter.$anonfun$apply$1$adapted(treeblobs.scala:30)
    at scala.collection.immutable.List.filterNot(List.scala:515)
    at scala.collection.immutable.List.filterNot(List.scala:79)
    at com.madgag.git.bfg.cleaner.FileDeleter.apply(treeblobs.scala:30)
    at com.madgag.git.bfg.cleaner.FileDeleter.apply(treeblobs.scala:29)
    at scala.Function$.$anonfun$chain$2(Function.scala:23)
    at scala.collection.LinearSeqOps.foldLeft(LinearSeq.scala:169)
    at scala.collection.LinearSeqOps.foldLeft$(LinearSeq.scala:165)
    at scala.collection.immutable.List.foldLeft(List.scala:79)
    at scala.Function$.$anonfun$chain$1(Function.scala:23)
    at com.madgag.git.bfg.cleaner.ObjectIdCleaner.$anonfun$cleanTree$1(ObjectIdCleaner.scala:125)
    at com.madgag.git.bfg.MemoUtil$$anon$3.load(memo.scala:74)
    at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
    at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
    at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
    ... 49 more