projectkudu / kudu

Kudu is the engine behind git/hg deployments, WebJobs, and various other features in Azure Web Sites. It can also run outside of Azure.
Apache License 2.0
3.12k stars 655 forks source link

Error getting zip file of "site/wwwroot": Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. #1448

Closed miparnisari closed 5 months ago

miparnisari commented 9 years ago

I'm getting this random error when downloading the ZIP file corresponding to "site/wwwroot". When the download succeeds, the file is around 20 MB.

This is the code I'm using: $apiUrl = "https://$websiteName.scm.azurewebsites.net/api/zip/site/wwwroot/" $method = "GET" $result = Invoke-RestMethod -Uri $apiUrl -Headers @{Authorization=("Basic {0}" -f $base64AuthInfo)} -Method $method -OutFile "wwwroot.zip" -TimeoutSec 600

The error is "Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host."

davidebbo commented 9 years ago

As a test, can you please try the following:

This should invoke basically the same zip API that you are using from PowerShell. It will be interesting to see if you get the same issue.

miparnisari commented 9 years ago

Yes, that works. I don't have any issues when downloading from a browser, it's just when using PowerShell.

I tried setting the user agent to [Microsoft.PowerShell.Commands.PSUserAgent]::Chrome but I'm getting the same error.

davidebbo commented 9 years ago

Strange that it would only happen from PowerShell. I usually do this using curl, e.g.

curl -o foo.zip %KUDU_URL%/api/zip/site/wwwroot/

Maybe worth trying. If that works reliably, then we'll have to blame PowerShell! :)

AWahlqvist commented 7 years ago

Found this issue through Google, having the same problem. It seems to happen when the site contains a lot of files, at least in my case.

It looks like it's tied to Invoke-RestMethod and Invoke-WebRequest though, it seems to work fine when using System.Net.WebClient instead, so that can be a workaround for this.

Basically: ` $WebClient = New-Object System.Net.WebClient $WebClient.Headers.Add('Authorization', 'Base 64 auth string...') $WebClient.Headers.Add('ContentType', 'multipart/form-data')

$WebClient.DownloadFile($Uri, $OutFile) `

davidebbo commented 7 years ago

That's odd. If it works reliably from the browser and using curl (as in the real curl, not the PowerShell alias), then I'll just blame it on something PowerShell is doing!

AWahlqvist commented 7 years ago

I tend to agree, the issue seems to exist in the built-in cmdlets Invoke-RestMethod and Invoke-WebRequest, not the Kudu API. Just added that here if someone else runs into this issue so they can see the workaround above that seems to run just fine.

But maybe this issue should be closed and a new one added to the PowerShell repo instead? I can add one there.

davidebbo commented 7 years ago

Yes, probably. However, to make it more convincing, it would be nice to have a repro of the PowerShell issue that doesn't involve Kudu. e.g. if you host the same zip somewhere else, does it happen with PowerShell? It could be that what makes it Kudu specific is that it can take a while for the bits to start flowing, as the zip has to first be created server side, and maybe that throws off PowerShell.

AWahlqvist commented 7 years ago

I'm pretty sure it wouldn't if the zip file was available already. What I've noticed is that the download slows down significantly when a folder contains a large number of files (I'm guessing it gets "streamed" while it's being packed?), and it's during this phase those cmdlets throws this error. So the download nearly finishes sometimes before this occurs.

This specific error usually means that the remote host actually sent a tcp RST to the client though, which would be a bit odd in this case?

davidebbo commented 7 years ago

Not sure. We just use System.IO.Compression.ZipArchive and System.Net.Http.PushStreamContent under the cover (see code here and here). And the file system is slower than a typical file system, so zipping a large tree is pretty slow.

sanjay3290 commented 4 years ago

its 2020 and i still face the same issue. Did somebody found a workaround?

AWahlqvist commented 3 years ago

@sanjay3290 : Not really, but we started to upload a PowerShell script that zips up the folder instead that also supports running "async" from the initial call so it doesn't get interrupted when the Azure front end cuts off the connection after 230 seconds. That has worked well for us, but not sure if it qualifies as a workaround.

ericthomas1 commented 3 years ago

2021 here...

I don't see a Site option in the Kudo console anymore. How do I download a .zip from here?

image

Also, hitting https://$FunctionAppName.scm.azurewebsites.net/api/zip/site/wwwroot/ yields a .zip file with only the Function's host.json file, no source code or support files.

image

How does one download the .zip for a Python Function!? (please don't say "FTP")

jvano commented 5 months ago

Hi

Kudu will continue to run in Azure App Service. However, this repo will no longer be maintained. If the problem persists and is related to running on Azure App Service, please open a support incident in Azure: https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request

This way we can better track and assist you on this case

Thanks,

Joaquin Vano Azure App Service