search

rtyley / bfg-repo-cleaner

Removes large or troublesome blobs like git-filter-branch does, but faster. And written in Scala
https://rtyley.github.io/bfg-repo-cleaner/
GNU General Public License v3.0
11.05k stars 545 forks source link

Exception with big repo and deletion of 18 blob ids #362

Open windwalker78 opened 4 years ago

windwalker78 commented 4 years ago 
  Hi,

  I'm trying to delete blobs by their ids and file list. I have checked that the user owns all files in the folder of the repository. This is the error I get:

This repo has been processed by The BFG before! Will prune repo before proceeding - to avoid unnecessary cleaning work on unused objects... Completed prune of old objects - will now proceed with the main job!

Found 101271 objects to protect Found 1443 tag-pointing refs : MASKED, ... Found 369 commit-pointing refs : MASKED, ...

Protected commits

These are your protected commits, and so their contents will NOT be altered:

Cleaning

Found 69396 commits Cleaning commits: 100% (69396/69396) Cleaning commits completed in 20,642 ms.

Updating 990 Refs

Ref                                                                                                  Before     After   
------------------------------------------------------------------------------------------------------------------------
refs/heads/master                                                                                  | b62d2162 | 9fe8fa46
refs/heads/*MASKED*                                                                                | c75022fd | ecdacf7e
refs/remotes/origin/*MASKED*                                                                       | ced17c7b | e21d2a9e
refs/remotes/origin/*MASKED*                                                                       | a59f91e7 | 3992159c
refs/remotes/origin/*MASKED*                                                                       | ffbccf67 | ba5ad774
refs/remotes/origin/*MASKED*                                                                       | e3bc58e9 | 580c19b5
refs/remotes/origin/*MASKED*                                                                       | f1b29fc0 | 8b96fd8c
refs/remotes/origin/*MASKED*                                                                       | 268db35b | 5fc12d31
refs/remotes/origin/*MASKED*                                                                       | 08ff011b | 74810f0a
refs/remotes/origin/*MASKED*                                                                       | 810ee9cb | 19e2e71e
refs/remotes/origin/*MASKED*                                                                       | 09361e36 | 30727b13
refs/remotes/origin/*MASKED*                                                                       | aceabf80 | bc771aee
refs/remotes/origin/*MASKED*                                                                       | b3d1745c | 5423eebf
refs/remotes/origin/*MASKED*                                                                       | 98ae9641 | 3e77e1d5
refs/remotes/origin/*MASKED*                                                                       | 6dc79613 | d344a6d8
 *MASKED*
...

Updating references: 100% (990/990) ...Ref update completed in 181 ms.

Commit Tree-Dirt History

Earliest                                              Latest
|                                                          |
...................DmmmDDmmmmmmmmmmmmmmmmmDDmmmmDDDmDmmmmmmm

D = dirty commits (file tree fixed)
m = modified commits (commit message or parents changed)
. = clean commits (no changes to file tree)

                        Before     After   
-------------------------------------------
First modified commit | 5965bb05 | 83d43aa8
Last dirty commit     | 6359c414 | ac33f8b3

Deleted files

Filename                            Git id                                                       
-------------------------------------------------------------------------------------------------
*MASKED*                          | 066122cc (951.9 MB)                                          
*MASKED*                          | a538e072 (951.9 MB)                                          
*MASKED*                          | 5a23ad9e (951.9 MB)                                          
*MASKED*                          | b4031251 (359.5 MB), 9e06c1cf (356.6 MB), 63b6c563 (342.4 MB)
*MASKED*                          | 377aef85 (219.1 MB)                                          
*MASKED*                          | 42c96777 (331.8 MB), c46811a6 (370.8 MB), ...                
*MASKED*                          | 40a1e3ae (373.2 MB)                                          
*MASKED*                          | b1be3bbc (185.1 MB), 0ffa5074 (185.7 MB), b1c37fc1 (747.6 MB)
*MASKED*                          | c897f0b4 (522.2 MB)                                          
*MASKED*                          | cb15b7c5 (264.7 MB), a9b7a3dc (290.2 MB)

java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at use.newer.java.Util.invokeMainMethod(Util.java:89) at use.newer.java.Util.executeMainProgram(Util.java:28) at use.newer.java.Util.checkJavaVersionAndExecuteMainProgram(Util.java:18) at use.newer.java.Version8.main(Version8.java:5) Caused by: java.io.IOException: No such file or directory at java.io.UnixFileSystem.createFileExclusively(Native Method) at java.io.File.createNewFile(File.java:1012) at scalax.io.support.FileUtils$.$anonfun$preOpen$1(FileUtils.scala:66) at scala.collection.immutable.List.foreach(List.scala:389) at scalax.io.support.FileUtils$.preOpen(FileUtils.scala:58) at scalax.io.support.FileUtils$.$anonfun$openOutputStream$1(FileUtils.scala:19) at scalax.io.managed.OutputStreamResource.nResource$1(OutputStreamResource.scala:38) at scalax.io.managed.OutputStreamResource.$anonfun$writableByteChannel$1(OutputStreamResource.scala:40) at scalax.io.managed.WritableByteChannelResource.nResource$2(WritableByteChannelResource.scala:35) at scalax.io.managed.WritableByteChannelResource.$anonfun$writer$1(WritableByteChannelResource.scala:37) at scalax.io.managed.WriterResource.open(WriterResource.scala:17) at scalax.io.Resource.$anonfun$acquireFor$1(Resource.scala:164) at scala.util.control.Exception$Catch.$anonfun$either$1(Exception.scala:248) at scala.util.control.Exception$Catch.apply(Exception.scala:224) at scala.util.control.Exception$Catch.either(Exception.scala:248) at scalax.io.Resource.acquireFor(Resource.scala:164) at scalax.io.Resource.acquireFor$(Resource.scala:160) at scalax.io.managed.WriterResource.acquireFor(WriterResource.scala:8) at resource.ManagedResourceOperations.apply(ManagedResourceOperations.scala:26) at resource.ManagedResourceOperations.apply$(ManagedResourceOperations.scala:26) at scalax.io.managed.WriterResource.apply(WriterResource.scala:8) at resource.ManagedResourceOperations.acquireAndGet(ManagedResourceOperations.scala:25) at resource.ManagedResourceOperations.acquireAndGet$(ManagedResourceOperations.scala:25) at scalax.io.managed.WriterResource.acquireAndGet(WriterResource.scala:8) at resource.ManagedResourceOperations.foreach(ManagedResourceOperations.scala:53) at resource.ManagedResourceOperations.foreach$(ManagedResourceOperations.scala:53) at scalax.io.managed.WriterResource.foreach(WriterResource.scala:8) at scalax.io.WriteChars.writeStrings(WriteChars.scala:78) at scalax.io.WriteChars.writeStrings$(WriteChars.scala:77) at scalax.io.managed.WriterResource.writeStrings(WriterResource.scala:8) at scalax.io.Output.writeStrings(Output.scala:125) at scalax.io.Output.writeStrings$(Output.scala:124) at scalax.file.FileOps.writeStrings(FileOps.scala:78) at com.madgag.git.bfg.cleaner.CLIReporter.reportFiles$1(Reporter.scala:229) at com.madgag.git.bfg.cleaner.CLIReporter.reportResults(Reporter.scala:243) at com.madgag.git.bfg.cleaner.RepoRewriter$.updateRefsWithCleanedIds$1(RepoRewriter.scala:140) at com.madgag.git.bfg.cleaner.RepoRewriter$.rewrite(RepoRewriter.scala:147) at com.madgag.git.bfg.cli.Main$.$anonfun$new$1(Main.scala:60) at com.madgag.git.bfg.cli.Main$.$anonfun$new$1$adapted(Main.scala:35) at scala.Option.map(Option.scala:146) at com.madgag.git.bfg.cli.Main$.delayedEndpoint$com$madgag$git$bfg$cli$Main$1(Main.scala:35) at com.madgag.git.bfg.cli.Main$delayedInit$body.apply(Main.scala:28) at scala.Function0.apply$mcV$sp(Function0.scala:34) at scala.Function0.apply$mcV$sp$(Function0.scala:34) at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12) at scala.App.$anonfun$main$1$adapted(App.scala:76) at scala.collection.immutable.List.foreach(List.scala:389) at scala.App.main(App.scala:76) at scala.App.main$(App.scala:74) at com.madgag.git.bfg.cli.Main$.main(Main.scala:28) at com.madgag.git.bfg.cli.Main.main(Main.scala) ... 8 more

Can you point me if I'm doing something wrong?

Kind regards

windwalker78 commented 4 years ago

It is actually a permissions issue, which pops-up during creation of the report, which is outside the folder of the repo. I suggest would be super helpful if you handle it and say something like: "The reports could not be saved to folder: /some/folder/repo.bfg-report"

Kind regards