search

jenkinsci / azure-artifact-manager-plugin

Jenkins Azure artifact manager plugin
https://plugins.jenkins.io/azure-artifact-manager/
MIT License
6 stars 10 forks source link

Large (~1GiB) file fails to upload #32

Open seansaleh opened 2 years ago

seansaleh commented 2 years ago

Version report

Jenkins and plugins versions report:

Click to expand! ``` Jenkins: 2.303.3 OS: Linux - 5.11.0-1022-azure --- ace-editor:1.1 analysis-model-api:10.8.0 ansicolor:1.0.1 antisamy-markup-formatter:2.5 apache-httpcomponents-client-4-api:4.5.13-1.0 authorize-project:1.4.0 azure-artifact-manager:97.v074e1332e88d azure-sdk:70.v63f6a95999a7 blueocean-commons:1.25.2 blueocean-rest:1.25.2 bootstrap4-api:4.6.0-3 bootstrap5-api:5.1.3-3 bouncycastle-api:2.25 branch-api:2.7.0 caffeine-api:2.9.2-29.v717aac953ff3 checks-api:1.7.2 cloudbees-folder:6.16 command-launcher:1.6 configuration-as-code:1.54 copyartifact:1.46.2 credentials:2.6.1 credentials-binding:1.27 data-tables-api:1.11.3-4 display-url-api:2.3.5 durable-task:493.v195aefbb0ff2 echarts-api:5.2.2-1 extended-choice-parameter:0.82 font-awesome-api:5.15.4-3 forensics-api:1.7.0 git:4.10.0 git-client:3.10.0 git-parameter:0.9.13 git-server:1.10 github:1.34.1 github-api:1.133 github-branch-source:2.11.3 github-checks:1.0.13 handlebars:3.0.8 jackson2-api:2.13.0-230.v59243c64b0a5 jaxb:2.3.0.1 jdk-tool:1.5 jjwt-api:0.11.2-9.c8b45b8bb173 job-dsl:1.78.1 jquery:1.12.4-1 jquery3-api:3.6.0-2 jsch:0.1.55.2 junit:1.53 leastload:3.0.0 lockable-resources:2.12 mailer:1.34 matrix-auth:2.6.8 matrix-project:1.19 momentjs:1.1.1 okhttp-api:4.9.2-20211102 p4:1.11.6 pipeline-build-step:2.15 pipeline-graph-analysis:1.12 pipeline-input-step:2.12 pipeline-milestone-step:1.3.2 pipeline-model-api:1.9.3 pipeline-model-definition:1.9.3 pipeline-model-extensions:1.9.3 pipeline-rest-api:2.19 pipeline-stage-step:2.5 pipeline-stage-tags-metadata:1.9.3 pipeline-stage-view:2.19 pipeline-utility-steps:2.11.0 plain-credentials:1.7 plugin-util-api:2.5.1 popper-api:1.16.1-2 popper2-api:2.10.2-1 reverse-proxy-auth-plugin:1.7.1 role-strategy:3.2.0 scm-api:2.6.5 script-security:1.78 snakeyaml-api:1.29.1 ssh-agent:1.23 ssh-credentials:1.19 ssh-slaves:1.33.0 sshd:3.1.0 startup-trigger-plugin:2.9.3 structs:308.v852b473a2b8c throttle-concurrents:2.5 timestamper:1.15 token-macro:267.vcdaea6462991 trilead-api:1.0.13 validating-string-parameter:2.8 warnings-ng:9.8.0 windows-azure-storage:365.vf41653c43b01 workflow-aggregator:2.6 workflow-api:2.47 workflow-basic-steps:2.24 workflow-cps:2633.v6baeedc13805 workflow-cps-global-lib:548.v9085a486966a workflow-durable-task-step:1101.vf832bc1ac745 workflow-job:2.42 workflow-multibranch:2.26 workflow-scm-step:2.13 workflow-step-api:2.24 workflow-support:3.8 ``` 
Ubuntu (Controller inside Docker) for controller and Windows Server 2019 for agent, started via SSH

Reproduction steps

Relevant build job config

archiveArtifacts artifacts: "game.zip, BuildReportSummary.json", onlyIfSuccessful: true, fingerprint: false
sleep 300 // By adding this I got many more error messages. Without this I only sometimes got error messages.

Relevant Jenkins CasC

unclassified:
  artifactManager:
    artifactManagerFactories:
    - azure:
        config:
          container: "jenkins"
          prefix: "jobs/"
          storageCredentialId: "build-storage-account"

Logs from general Jenkins log: (repeated many times)

Click to expand! ``` Dec 01, 2021 1:26:47 PM WARNING hudson.model.Run getArtifactsUpTo null java.lang.NullPointerException: Container must not be null at java.base/java.util.Objects.requireNonNull(Objects.java:246) at com.microsoft.jenkins.artifactmanager.AzureBlobVirtualFile.run(AzureBlobVirtualFile.java:110) Caused: java.io.IOException at com.microsoft.jenkins.artifactmanager.AzureBlobVirtualFile.run(AzureBlobVirtualFile.java:125) at hudson.model.Run.getArtifactsUpTo(Run.java:1159) at com.cloudbees.workflow.rest.external.RunExt.createMinimal(RunExt.java:251) at com.cloudbees.workflow.rest.external.RunExt.createNew(RunExt.java:315) at com.cloudbees.workflow.rest.external.RunExt.create(RunExt.java:307) at com.cloudbees.workflow.rest.external.JobExt.create(JobExt.java:143) at com.cloudbees.workflow.rest.endpoints.JobAPI.doRuns(JobAPI.java:69) at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:710) at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:393) at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:405) at com.cloudbees.workflow.util.ServeJson$Processor.invoke(ServeJson.java:30) at org.kohsuke.stapler.PreInvokeInterceptedFunction.invoke(PreInvokeInterceptedFunction.java:26) at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:208) at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:141) at org.kohsuke.stapler.MetaClass$11.doDispatch(MetaClass.java:536) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:766) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:898) at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:457) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:766) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:898) at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:281) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:766) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:898) at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:281) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:766) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:898) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:694) at org.kohsuke.stapler.Stapler.service(Stapler.java:240) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:156) at jenkins.telemetry.impl.UserLanguages$AcceptLanguageFilter.doFilter(UserLanguages.java:128) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:153) at jenkins.security.ResourceDomainFilter.doFilter(ResourceDomainFilter.java:80) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:153) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:159) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:159) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:92) at org.jenkinsci.plugins.reverse_proxy_auth.ReverseProxySecurityRealm$1.doFilter(ReverseProxySecurityRealm.java:559) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:92) at jenkins.security.AcegiSecurityExceptionFilter.doFilter(AcegiSecurityExceptionFilter.java:52) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:53) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:121) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:115) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97) at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:105) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97) at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:133) at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:92) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97) at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:218) at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:212) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97) at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:97) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:62) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:109) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:109) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:51) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:85) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at jenkins.security.SuspiciousRequestFilter.doFilter(SuspiciousRequestFilter.java:35) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:578) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:386) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Thread.java:829) ```

Logs from loggers ALL jenkins.model.StandardArtifactManager com.microsoft.jenkins.artifactmanager.AzureBlobVirtualFile jenkins.model.ArtifactManager

Click to expand! ``` Dec 01, 2021 1:25:45 PM FINE jenkins.model.ArtifactManager Archiving from G:\j\NHP_Release\out: {BuildReportSummary.json=BuildReportSummary.json, game.zip=game.zip} Dec 01, 2021 1:25:50 PM FINE com.microsoft.jenkins.artifactmanager.AzureBlobVirtualFile enter cache null / jobs/myjobfolder/Myjobname/22/artifacts Dec 01, 2021 1:25:50 PM FINE com.microsoft.jenkins.artifactmanager.AzureBlobVirtualFile enter cache null / jobs/myjobfolder/Myjobname/22/artifacts [cut many identical log entries] Dec 01, 2021 1:30:54 PM FINE com.microsoft.jenkins.artifactmanager.AzureBlobVirtualFile enter cache jenkins / jobs/myjobfolder/Myjobname/22/artifacts Dec 01, 2021 1:30:54 PM FINE com.microsoft.jenkins.artifactmanager.AzureBlobVirtualFile using cache jenkins / jobs/myjobfolder/Myjobname/22/artifacts: 1 file entries Dec 01, 2021 1:30:54 PM FINER com.microsoft.jenkins.artifactmanager.AzureBlobVirtualFile cache hit on listing of jenkins / jobs/myjobfolder/Myjobname/22/artifacts Dec 01, 2021 1:30:54 PM FINER com.microsoft.jenkins.artifactmanager.AzureBlobVirtualFile cache hit on file status of jenkins / jobs/myjobfolder/Myjobname/22/artifacts/BuildReportSummary.json Dec 01, 2021 1:30:54 PM FINER com.microsoft.jenkins.artifactmanager.AzureBlobVirtualFile cache hit on length of jenkins / jobs/myjobfolder/Myjobname/22/artifacts/BuildReportSummary.json Dec 01, 2021 1:30:54 PM FINER com.microsoft.jenkins.artifactmanager.AzureBlobVirtualFile cache hit on directory status of jenkins / jobs/myjobfolder/Myjobname/22/artifacts/BuildReportSummary.json Dec 01, 2021 1:30:54 PM FINER com.microsoft.jenkins.artifactmanager.AzureBlobVirtualFile cache hit on directory status of jenkins / jobs/myjobfolder/Myjobname/22/artifacts/BuildReportSummary.json Dec 01, 2021 1:30:54 PM FINE com.microsoft.jenkins.artifactmanager.AzureBlobVirtualFile exit cache jenkins / jobs/myjobfolder/Myjobname/22/artifacts ```

Results

Expected result:

Large file (game.zip, size 960 MiB) is uploaded to Azure, and so is small file (BuildReportSummary.json, size 1KiB)

Actual result:

Build passes, but doesn't upload large file. Large file doesn't exist in Azure anywhere, nor in Jenkins web UI. Small file is uploaded.

timja commented 2 years ago

Thanks, no obvious cause there, will try reproduce when I get time but quite busy currently.