microsoft / windows-dev-center-vsts-extension

Visual Studio Team Services (VSTS) extension for performing continuous delivery to the Windows Dev Center store from your automated CI builds
Other
42 stars 17 forks source link

Timeout Uploading Test Package #46

Open JKennedy24 opened 7 years ago

JKennedy24 commented 7 years ago

I am using the task to upload a package to test but it timed out after 10 minutes Log: 2017-11-14T13:06:00.5701197Z Creating zip file... 2017-11-14T13:06:06.0545101Z Uploading zip file... 2017-11-14T13:16:02.2041902Z ##[error]Failed to upload file! Error = StorageError: Operation could not be completed within the specified time. 2017-11-14T13:16:02.2041902Z RequestId:8d7239b9-001e-00ac-804a-5da9c7000000 2017-11-14T13:16:02.2041902Z Time:2017-11-14T13:16:02.1008004Z 2017-11-14T13:16:02.2041902Z ##[error]Failed to upload file! Error = StorageError: Operation could not be completed within the specified time.

Is it possible to increase the timeout? or is this actually another issue.

My vsts agent is local and my UWP package is 60mb

zurdev commented 7 years ago

The timeout is hard-coded and cannot be changed from the task inputs. The actual file being uploaded includes the APPXBundle and metadata (if provided). You could locate the file under $(agent.workFolder)\temp.zip. Could you share the zip file size and the Azure blob URL (from the task log)? You might be trying to upload the zip file to a remote region, or if your agent is behind a proxy, it could significantly slowdown the upload bandwidth. Would it work for you to spin up an Azure VM and install VSTS agent in the cloud? Sorry for the inconvenience.

JKennedy24 commented 7 years ago

size of the temp.zip is 60mb

My agent has 2mb/s upload so I would estimate 30 seconds - 1 minute tops and there is no proxy on my agent machine.

I can't seem to find the azure upload url in my log. If you let me know an email address I will happily email the log file to you

I could potentially use some of my vsts minutes to use a cloud agent machine for release

Let me know if you would like me to email across the log file

zurdev commented 7 years ago

For the Azure blob URL, you should enable debug logging by setting the release variable system.debug to true. If you could try and repro from Azure VM that would help verifying whether the timeout is a result of slow connectivity. Thanks for your patience and cooperation.

JKennedy24 commented 7 years ago

@zurdev I will try that during my next release cycle.

I think potentially the timeout isn't long enough. I can see in the logs the upload completes after 11 mins 30 seconds

So it's probably that the timeout is just 1 min 30 too short.

Could you potentially increase the timeout to 15 mins?

zurdev commented 7 years ago

There's more than one timeout involved. The max execution timeout is set to 15min already, so clearly you're not hitting this timeout. We also use Exponential Retry Policy, to retry the upload to blob storage, and we're not changing the defaults, which are the recommended values. I'm assuming the agent machine you're running from is remote from the Azure blob storage account, and that creates the problem. I still recommend that you try to deploy an agent in an Azure VM that is located in the US region.

JKennedy24 commented 7 years ago

@zurdev Tried on a Hosted VSTS machine and still got the same timeout.

Interrestingly this release I turned on Debug logging and the Upload succeeded but the Task timed out Polling Flight Submission

Looks like Polling Flight Submission - Certification.

This explains why it times out, I would never expect the Certification process to complete in under 10 minutes.

I would imagine putting Skip polling would fix the issue

zurdev commented 7 years ago

Glad that running the task from a hosted VSTS machine resolved the upload timeout. As for the task timeout while polling for submission to complete - this timeout is enforced by the VSTS agent based on the task Control Options -> Timeout settings. You may either check the Skip Polling option, or remove set the timeout to 0.

JKennedy24 commented 7 years ago

@zurdev I would like to be able to release the app using my agent. Is it possible to increase the timeout?

JKennedy24 commented 6 years ago

any update on whether you are able to increase the hardcoded timeout?

ValonK commented 3 years ago

We are experiencing the exact same issue, our UWP Package is 35Mb big

##[error]Failed to upload file! Error = StorageError: Operation could not be completed within the specified time.
##[debug]Processed: ##vso[task.issue type=error;]Failed to upload file! Error = StorageError: Operation could not be completed within the specified time.

Is there a way to use our own local Agent, or set the Timeout manually? We cannot use Hosted Agents :(