jenkinsci / last-changes-plugin

https://plugins.jenkins.io/last-changes
https://plugins.jenkins.io/last-changes/
31 stars 30 forks source link

Can't delete .git directory after build, exception thrown #49

Closed loonychewy closed 5 years ago

loonychewy commented 6 years ago

Hi,

I'm trying to get the latest commit id from my git repo, if new commits are made since the last time the pipeline job was executed. Here's my scriptable pipeline script. (My git repo is checked-out to a subfolder "MY_REPO" of the jenkins workspace.)

node
{
    stage("Checkout")
    {
        dir ("MY_REPO")
        {
            git url: user@gitrepo:/srv/git/MY_REPO.git, branch: "master", credentialsId: "12345678-1234-1234-1234-123456789012"
        }

        def publisher = LastChanges.getLastChangesPublisher "LAST_SUCCESSFUL_BUILD", "SIDE", "NONE", false, false, "", "", "", "MY_REPO", ""
        publisher.publishLastChanges()
        def changes = publisher.getLastChanges()
        if (changes.getCommits().size() > 0)
        {
            def currentRevision = changes.getCurrentRevision()
            echo "Latest CommitId: " + currentRevision.commitId
        }
        else
        {
            echo "No new commits since last successful build"
        }
    }
}

But I get an exception at the end of the job execution, where it says it can't delete a file in the .git folder. I checked using ProcessExplorer, the file is still being locked by Jenkins process itself. I'm running the WAR version of jenkins using "java -jar jenkins.war". (Edit:) I'm using the pipeline (non-multibranch) type of Jenkins project However, if I use "PREVIOUS_REVISION" instead of "LAST_SUCCESSFUL_BUILD", I do not get this exception. My guess is that this plugin is still holding on to the file in .git folder when it is calling vcsTargetDir.deleteRecursive()? Or did I do something incorrectly here? The full exception stack is below:

java.nio.file.FileSystemException: D:\Jenkins\jobs\MY_JOB\builds\210\.git\objects\pack\pack-a76739f8dc0c4b6f3eb19e7e54c152b7cc72271f.pack: The process cannot access the file because it is being used by another process.

    at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
    at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
    at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
    at sun.nio.fs.WindowsFileSystemProvider.implDelete(Unknown Source)
    at sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(Unknown Source)
    at java.nio.file.Files.deleteIfExists(Unknown Source)
    at hudson.Util.tryOnceDeleteFile(Util.java:316)
    at hudson.Util.deleteFile(Util.java:272)
    at hudson.FilePath.deleteRecursive(FilePath.java:1270)
    at hudson.FilePath.deleteContentsRecursive(FilePath.java:1279)
    at hudson.FilePath.deleteRecursive(FilePath.java:1261)
    at hudson.FilePath.deleteContentsRecursive(FilePath.java:1279)
    at hudson.FilePath.deleteRecursive(FilePath.java:1261)
    at hudson.FilePath.deleteContentsRecursive(FilePath.java:1279)
    at hudson.FilePath.deleteRecursive(FilePath.java:1261)
    at hudson.FilePath.access$1200(FilePath.java:209)
    at hudson.FilePath$DeleteRecursive.invoke(FilePath.java:1241)
    at hudson.FilePath$DeleteRecursive.invoke(FilePath.java:1237)
    at hudson.FilePath.act(FilePath.java:1047)
    at hudson.FilePath.act(FilePath.java:1025)
    at hudson.FilePath.deleteRecursive(FilePath.java:1235)
    at com.github.jenkins.lastchanges.LastChangesPublisher.perform(LastChangesPublisher.java:292)
    at com.github.jenkins.lastchanges.pipeline.LastChangesPublisherScript.doPublish(LastChangesPublisherScript.java:31)
    at com.github.jenkins.lastchanges.pipeline.PublishLastChangesStep$Execution.run(PublishLastChangesStep.java:43)
    at com.github.jenkins.lastchanges.pipeline.PublishLastChangesStep$Execution.run(PublishLastChangesStep.java:24)
    at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousStepExecution.start(AbstractSynchronousStepExecution.java:42)
    at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:229)
    at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:153)
    at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
    at com.github.jenkins.lastchanges.pipeline.LastChangesPublisherScript.publishLastChanges(LastChangesPublisherScript.java:38)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
    at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:47)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157)
    at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:133)
    at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155)
    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159)
    at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
Caused: java.io.IOException: Unable to delete 'D:\Jenkins\jobs\MY_JOB\builds\210\.git\objects\pack\pack-a76739f8dc0c4b6f3eb19e7e54c152b7cc72271f.pack'. Tried 3 times (of a maximum of 3) waiting 0.1 sec between attempts.
    at hudson.Util.deleteFile(Util.java:277)
    at hudson.FilePath.deleteRecursive(FilePath.java:1270)
    at hudson.FilePath.deleteContentsRecursive(FilePath.java:1279)
    at hudson.FilePath.deleteRecursive(FilePath.java:1261)
    at hudson.FilePath.deleteContentsRecursive(FilePath.java:1279)
    at hudson.FilePath.deleteRecursive(FilePath.java:1261)
    at hudson.FilePath.deleteContentsRecursive(FilePath.java:1279)
    at hudson.FilePath.deleteRecursive(FilePath.java:1261)
    at hudson.FilePath.access$1200(FilePath.java:209)
    at hudson.FilePath$DeleteRecursive.invoke(FilePath.java:1241)
    at hudson.FilePath$DeleteRecursive.invoke(FilePath.java:1237)
    at hudson.FilePath.act(FilePath.java:1047)
    at hudson.FilePath.act(FilePath.java:1025)
    at hudson.FilePath.deleteRecursive(FilePath.java:1235)
    at com.github.jenkins.lastchanges.LastChangesPublisher.perform(LastChangesPublisher.java:292)
    at com.github.jenkins.lastchanges.pipeline.LastChangesPublisherScript.doPublish(LastChangesPublisherScript.java:31)
    at com.github.jenkins.lastchanges.pipeline.PublishLastChangesStep$Execution.run(PublishLastChangesStep.java:43)
    at com.github.jenkins.lastchanges.pipeline.PublishLastChangesStep$Execution.run(PublishLastChangesStep.java:24)
    at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousStepExecution.start(AbstractSynchronousStepExecution.java:42)
    at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:229)
    at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:153)
    at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
    at com.github.jenkins.lastchanges.pipeline.LastChangesPublisherScript.publishLastChanges(LastChangesPublisherScript.java:38)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
    at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:47)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157)
    at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:133)
    at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155)
    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159)
    at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
    at WorkflowScript.run(WorkflowScript:146)
    at ___cps.transform___(Native Method)
    at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)
    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixName(FunctionCallBlock.java:77)
    at sun.reflect.GeneratedMethodAccessor248.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
    at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
    at com.cloudbees.groovy.cps.Next.step(Next.java:83)
    at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
    at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
    at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122)
    at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261)
    at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
    at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$101(SandboxContinuable.java:34)
    at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.lambda$run0$0(SandboxContinuable.java:59)
    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
    at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:58)
    at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:174)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232)
    at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
rmpestano commented 6 years ago

It is probably related to #46, if you or @KLBonn want to hack I would accept proposed solution in #46 or an additional parameter to skip clean up. Just sent a PR and I can review and or guide you. There is a "running this plugin locally" section on README so you can test your changes.

loonychewy commented 6 years ago

Thanks for your reply, @rmpestano. I might give the proposed solution a try just to see if it works, will let you know again if I get to it.

Edit: I probably can't set this issue as duplicate cos I don't have write access. https://help.github.com/articles/about-duplicate-issues-and-pull-requests/ Please feel free to do it for me.

loonychewy commented 6 years ago

I tried the suggested solution here https://github.com/jenkinsci/last-changes-plugin/issues/46#issuecomment-395797467 , but it did not fix the problem. One more point to add is that this issue only occurs for me when I'm using LAST_SUCCESSFUL_BUILD, and there are no commits to the git repo from the last jenkins build and the current one.

Also, I noticed in LastChangesPublisher.java, line 281, the parameters for the resultMessage for "from revision" is lastChanges.getCurrentRevision, and "to revision" is lastChanges.getPreviousRevision, when it should be the other way round, am I wrong?

rmpestano commented 6 years ago

Hi @loonychewy,

I probably can't set this issue as duplicate cos I don't have write access.

Don't worry about that.

Also, I noticed in LastChangesPublisher.java, line 281, the parameters for the resultMessage for "from revision" is lastChanges.getCurrentRevision, and "to revision" is lastChanges.getPreviousRevision, when it should be the other way round, am I wrong?

sincerely I don't know if the "FROM" revision should be before or after, what's important is that we're diffing two revisions. Makes sense?

Do you need any help on fixing the issue? anything on code that you didn't understand?

loonychewy commented 6 years ago

Yes please, some help would be greatly appreciated because I'm not too familiar with git and jenkins. I'm suspecting something in GitLastChange.java or the classes it is using is still holding on to the file in the .git folder when we try to delete the folder.

As for the from and to revision, it might confuse those who are reading the log output, at least it seems so to me...

fm0195 commented 5 years ago

Hello, I am facing the same issue as above. Can a try-catch block be used to print a warning instead of trowing an exception when deleting the copied files?

            ```
           try {
                vcsTargetDir.deleteRecursive();//delete copied dir on master
            } catch(Exception e) {
                listener.error("WARN: Could not delete copied files to master.");
            }
rmpestano commented 5 years ago

Hi @fm0195 , can you try with last-changes version below? This zip contains the fix from this PR #63, just unzip and upload/install the plugin on your jenkins instalation:

last-changes.zip

If we confirm it solves the problem I'll release a new version with the fix.

cutik commented 5 years ago

Hi @fm0195 , can you try with last-changes version below? This zip contains the fix from this PR #63, just unzip and upload/install the plugin on your jenkins instalation:

last-changes.zip

If we confirm it solves the problem I'll release a new version with the fix.

@rmpestano I can confirm that version from archive solves the issue, thanks for the solution!

rmpestano commented 5 years ago

v2.7.0 was released containing the fix.

Thank you guys.