Publishing a ZIP package does not succeed and breaks subsequent publish attempts

vladislav-mitev commented 6 years ago

NB: This issue was not reproducible up until 20.04.2018. We noticed this issue happening from 25.04.2018.

When a function app is created (via Azure Portal or with ARM template deployment) initially the publishing from VS2017 succeeds with no worries. If we then try to publish a ZIP package, it fails and it causes all subsequent publish attempts to fail. We tried different approaches to publishing the ZIP package: msdeploy, ZIP push url and msdeploy extension within the deployment template. All of them seem to fail with similar error: internal server IO error saying that it is not able to create/access some directories. At this point if we connect via FTP client, we are able to freely modify the file/dir structure and upload all required contents. If we than again try to publish (ZIP or through VS2017), we again receive the same error. In all these cases when the publish process fails with IO error, we see that (1) all prior existing files and directories have been deleted (which is correct) and (2) only some files and directories uploaded. It seems that the error happens in the middle of file upload process.

Investigative information

Timestamp: 25.04.2018 - 27.04.2018
Function App version: 1.0
Function App name: CustomerEventsQueueProcessor-events-customer001
Function name(s) (as appropriate): EventQueueProcessor
Region: West Europe

Repro steps and Actual behavior

Create a Function App hosted under Consumption plan A) through Azure Portal OR B) using template deployment
Try to publish the contents through VS2017. This should succeed
Try to publish a ZIP package A) Use ZIP PUSH DEPLOYMENT as described here https://docs.microsoft.com/en-us/azure/azure-functions/deployment-zip-push OR B) use msdeploy CMD tool OR C) use MSDeploy extension resource within the ARM template of the Function App with a URL to the ZIP in an Azure Storage Account
The publish process fails with the following error respectively, but some files and folders were published: A) Internal Server Error 503 Service Unavailable OR B) The remote server responded with a generic 500 Internal Server Error OR C) 403 Forbidden Response with an Inner Exception message saying "System.IO.DirectoryNotFoundException: Could not find a part of the path 'D:\home\site\wwwroot\bin\runtimes\osx\"
Trying to publish from VS2017 without deleting existing files results in this error: "An error was encountered when processing operation 'Create Directory' on 'D:\home\site\wwwroot\bin\runtimes'"

Expected behavior

Publish of a ZIP package to succeed and not break subsequent publish attempts.

Known workarounds

Publish app content without ZIP-ing it

Related information

Programming language used: C# .NET 461
Bindings used: Queue Trigger bound with: public static async Task Run( [QueueTrigger("%PullQueueName%", Connection = "StorageConnectionString")] CloudQueueMessage queueMessage, ILogger log)

paulbatum commented 6 years ago

@watashiSHUN I know there is functions specific code in our deployment handling, any idea if this could be related to that?

sweetest commented 6 years ago

Facing the same problem, is there any workaround at least?

vladislav-mitev commented 6 years ago

Yes, there is but a bit more manual. It works however. You should directly use the msdeploy command line tool. You have to download the publish profile of your web app to gain the credentials and have your app files unzipped. Personally, I'm using this tool which wraps the raw msdeploy usage and works great: https://github.com/davidebbo/WAWSDeploy

Azure / azure-functions-host