Azure Web App Deployment error: Web Deploy cannot modify the file - ERROR_FILE_IN_USE

[AndreaPic]

Hi, I'm using Azure Web App Deployment Task in a step of continuous integration on Visual Studio Team Services in hosted environment. Nightly are scheduled many rebuild of different app in a different build definition. Every night some of this build definition goes in error in Web Deploy step whit this message:

Web Deploy cannot modify the file 'X' on the destination because it is locked by an external process. In order to allow the publish operation to succeed, you may need to either restart your application to release the lock, or use the AppOffline rule handler for .Net applications on your next publish attempt. Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_FILE_IN_USE.

The file in use isn't ever the same but can change. Can you help me ? What can I do ? There is a way to force stopping and restarting the app ? Thank you Andrea

[yaananth]

could you try setting donotdelete flag?

[AndreaPic]

I've set the "do not delete" flag and tomorrow I'll to see the output. But I've a question about this flag. What are "additional files" ? Thanks Andrea

[AndreaPic]

Hi @yaananth , In the last nightly build with "do not delete" flag active the problem did not occur. I'll see the next nightly build and if it will still ok the problem could be solved. Thanks Andrea

[AndreaPic]

Hi @yaananth , Past two nightly build and complete continuous integration has the same problem. It seems that donotdelete flag didn't solve the problem. The last problem is :

Web Deploy cannot modify the file 'Microsoft.CodeAnalysis.CSharp.dll' on the destination because it is locked by an external process. In order to allow the publish operation to succeed, you may need to either restart your application to release the lock, or use the AppOffline rule handler for .Net applications on your next publish attempt. Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_FILE_IN_USE

Have you another idea ? Thanks

[codito]

@AndreaPic this issue is similar to #1233, can you please try the workaround mentioned there?

Publish-AzureWebsite doesn't support AppOffline option, the newer msdeploy based azure website deployment task will fix this issue.

[AndreaPic]

Hi @codito @yaananth I've tried the solution mentioned but I've the "login" problem too. Is the support for AppOffline option planned for "Publish-AzureWebsite" Task ?

[codito]

@AndreaPic sorry, there isn't a way to fix AppOffline issue with this task. Can you please try the msdeploy based website deployment task? It is in preview at the moment, however you can use tfx-cli to upload the task to your account and use it. We will be glad to hear your feedback.

[codito]

Just realized your account may not be enrolled into preview. Kindly drop an email to rm_customer_queries on the domain microsoft dot com if you don't see the task. /cc:@roopeshnair

[tebeco]

@codito is the issue fixed ? It's marked as closed but still occurs from vsts release

I'd like to keep release managment instead of powershell publishing if possible

I guess this issue should NOT be closed right ?

[codito]

@tebeco this was closed because there is an msdeploy based alternative for this task available which supports taking app offline. Can you please try AzureRMWebAppDeployment?

There is no plan to support app offline in this task.

[tebeco]

A ok. No pb ;)

Thx for the precision

[sebvst]

Even if I enable "take app offline" the error occurs. In my case it is always msvcr100.dll that causes the problem.

Does anyone else have the same problem?

[knoxitus]

@sebvst I have exactly the same problem that msvcr100.dll is locked by using the AzureRMWebAppDeployment build step, . Enbling or disabling "take app offline" has no effect.

[asranja]

can you mail the release logs to RM_Customer_Queries@microsoft.com. Make sure to keep the system.debug variable to true and enable the app offline feature.

[KrishnaAdityaB]

Please set below setting on the app. MSDEPLOY_RENAME_LOCKED_FILES = 1 Automating this is in backlog. Marking this thread as enhancement

[dpodder]

@KrishnaAdityaB @RoopeshNair I've started hitting ERROR_FILE_IN_USE recently when deploying to staging slot using AzureRmWebAppDeployment (take app offline is set). Oddly does not happen when deploying directly to production; only happens when deploying to a slot.

Could you clarify what MSDEPLOY_RENAME_LOCKED_FILES = 1 is expected to do? Do the old files get deleted, after a rename? Alternatively, would you recommend adding a manual step to take the app offline prior to executing the deployment task?

[nickspiers]

I have the take app offline option selected, am not deploying to a slot but directly to production and am getting the ERROR_FILE_IN_USE.

[YanivDa]

@asranja @KrishnaAdityaB MSDEPLOY_RENAME_LOCKED_FILES = 1 solved the issue. Is this workaround a permanent solution? what is the effect of this parameter?

[erinha]

The VSTS Azure App Service Deploy task fails occasionally with ERROR_FILE_IN_USE when deploying to a deployment slot and unfortunately using MSDEPLOY_RENAME_LOCKED_FILES=1 did not solve the issue for me. Checking "Take App Offline" also doesn't prevent the failure. The only solution I've found so far that works is to have an Azure PowerShell task run before the deployment to stop the service slot and then restart it again after the deployment. This doesn't seem expected though, based on the previous comments.

Can someone please provide more clarity about what the MSDEPLOY_RENAME_LOCKED_FILES setting does, and how it should be used?

[rajatagrawal-dev]

Hi @erinha

Can you provide us with the debug/verbose logs? We are in the process of forwarding this issue to the Azure team and your logs will help them in debugging it. By setting MSDEPLOY_RENAME_LOCKED_FILES=1 in the Azure App Settings, WebDeploy will attempt to rename the DLL if it can’t be overwritten. Azure team is better equipped to answer these questions in detail. So please send us the debug logs at RM_Customer_Queries at Microsoft dot com.

To get the debug logs please add a variable "system.debug" in your release definition and set its value to "true". Triggering a release then will give you the release logs.

[erinha]

Sure, I can send logs.

There are a couple of problems though:

1. The issue is intermittent (if I trigger the release deployment manually, it does not occur). I have to wait for a triggered deployment to occur (based on when someone on our team pushes to master).
2. I've currently worked around the issue by adding an Azure PowerShell task to stop the service before deploying to the slot and another after the deployment to start the service. So with this in place I don't see the error.

I'm assuming that the debug logs are only useful when the error occurs, so I'll enable debug logs for the definition and then have to disable my workarounds and wait for a failure.

[erinha]

FYI @rajatagrawal-dev, I was able to capture logs and sent them to the RM_Customer_Queries at Microsoft dot com email address, as you instructed. Hopefully they'll help diagnose the issue!

[jdshkolnik]

This is also happening with TFS 2017.

[onionhammer]

Yeah I'm using the new MS deploy task in release management VSTS, and I'm seeing this issue. I'm using the 'take app offline' flag and I'm using the MSDEPLOY_RENAME_LOCKED_FILES=1 setting.

This is the error I'm seeing. Could it be a byproduct of using the 'always online' setting for the web app? Also this is a aspnet core (on full .net framework) app.

2016-12-22T15:33:04.8639698Z Info: Using ID '755cb0bc-77e3-49eb-a055-b7342067560f' for connections to the remote server.

2016-12-22T15:33:06.7768978Z Info: Using ID 'a21a4759-da36-4412-ac68-fda97671b780' for connections to the remote server.

2016-12-22T15:33:11.2419112Z Info: Updating file (prod-c2c-borrowers\CRF.Connect2Capital.BorrowerWeb.exe).

2016-12-22T15:33:13.0488909Z ##[debug]rc:4294967295

2016-12-22T15:33:13.0488909Z ##[debug]rc:4294967295

2016-12-22T15:33:13.0488909Z ##[debug]success:false

2016-12-22T15:33:13.0488909Z ##[debug]success:false

2016-12-22T15:33:13.0608910Z ##[error]Failed to deploy website.

2016-12-22T15:33:13.0608910Z ##[debug]Processed: ##vso[task.issue type=error;]Failed to deploy website.

2016-12-22T15:33:13.0618962Z ##[debug]System.DefaultWorkingDirectory=C:\a\r1\a

2016-12-22T15:33:13.0618962Z ##[error]Error Code: ERROR_FILE_IN_USE

2016-12-22T15:33:13.0618962Z ##[debug]Processed: ##vso[task.issue type=error;]Error Code: ERROR_FILE_IN_USE

2016-12-22T15:33:13.0618962Z More Information: Web Deploy cannot modify the file 'CRF.Connect2Capital.BorrowerWeb.exe' on the destination because it is locked by an external process.  In order to allow the publish operation to succeed, you may need to either restart your application to release the lock, or use the AppOffline rule handler for .Net applications on your next publish attempt.  Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_FILE_IN_USE.

2016-12-22T15:33:13.0618962Z Error count: 1.

2016-12-22T15:33:13.0618962Z 

[mxa0079]

My team is also running into this issue (even after setting the flag mentioned in this thread... when we first added the flag the deployment started to work but this morning is failing again).

This is preventing us from using our Release Pipeline to publish to production, which has a signifcant impact in our productivity.

Could Microsoft please confirm that there is active work on this issue? Has a root cause been identified? Do we have an expected ETA? Any logs we can provide to help solve the issue?

Thanks!

[Adam-S-Daniel]

And with TFS 2012 deploying to Windows Server 2012. Locked file is msvcr100.dll. Fix is to restart app pool. Restarting appdomain does not resolve. EnableMSDeployAppOffline=true does not resolve, but it does cause the application to go offline until someone manually deletes app_offline.htm, which is, ahem, quite problematic.

[moozzyk]

Linking https://github.com/aspnet/Home/issues/694 which seems to be related.

[pgmolloy]

This problem just started for me on my normal deployment process. Could we back up to the previous version of whatever was modified in the last few days. Having to stop and restart the webapp every time even in development debug mode is getting very inefficient.

[flagbug]

Same happening here since just now

[Eneuman]

I'm having this issue as well now

[evermeire]

Same here.

[minidfx]

I use teamcity for deploying my app on Azure and I've the same error even with the MSDEPLOY_RENAME_LOCKED_FILES set to 1.

[moozzyk]

https://github.com/aspnet/Home/issues/694#issuecomment-269997757

[alex-oswald]

Same problem. MSDEPLOY_RENAME_LOCKED_FILES set to 1 worked for me. In my case it was the application dll that was causing the issue.

[bhenn]

Same here.

[NBelham]

Also seeing the same issue in the past 20 days approx. Using VS2015 Web Project Publish which includes this WebDeploy parameter -enablerule:AppOffline but I am finding that the web application isn't going offline and hence the web publish fails with the error message:

Web Deploy cannot modify the file 'someName.dll' on the destination because it is locked by an external process. In order to allow the publish operation to succeed, you may need to either restart your application to release the lock, or use the AppOffline rule handler for .Net applications on your next publish attempt. Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_FILE_IN_USE.

NOTE: Setting MSDEPLOY_RENAME_LOCKED_FILES set to 1 will cause the Web App to restart and so the first publish afterwards should work, but there's no guarantee your publish will beat another user visiting your site and then causing the Web Deploy to fail because of a lock assembly!

[zhangpengchen]

Same here

[JoshuaPHolden]

Same problem here. The deployment task is set to take offline but it never works unless I manually stop the application in the portal, run the build/deployment and then turn the site back on.

[ArashMotamedi]

Started experiencing this on my Asp.net core web app just now as well. The .exe file is locked and can't be overwritten.

For reference, this is the deployment command: Executing command ["C:\...\msdeploy.exe" -source:manifest='C:\...\SourceManifest.xml' -dest:manifest='C:\...\DestinationManifest.xml',ComputerName='https://....scm.azurewebsites.net/msdeploy.axd?site=...',UserName='...',Password='...',IncludeAcls='False',AuthType='Basic' -verb:sync -enablerule:AppOffline -retryAttempts:20]

[vincent1173]

@bhenn , @NBelham , @zhangpeng-kooboo , @nezoic , @ArashMotamedi , FILE_IN_USE issue occurs when DLL’s are locked. App_offline is believed to bring the app service down and unlock the resource. But the latest ASP.NET Core module expects app_offline.htm in root folder, but MSDeploy provides App_Offline.htm( case-sensitive). This issue is already in an active github issue with ASP.net repo and they informed that it will be fixed soon.

You can stop the App Service, Deploy the app and then start the app service to avoid FILE_IN_USE issue using Azure App Service Manage Task.

[kirkone]

@vincentdass maybe I did something wrong, but the Azure App Service Manage Task only provides the possibility to switch deployment slots for me.

Thanks, KirK

[ArashMotamedi]

@vincentdass Thanks. Could you please elaborate if there are any differences in the deployment behavior of Asp.net Core apps vs. Asp.net apps? My understanding is that when a classic (non-core) Asp.net app was deployed, IIS would simply react to the presence of new files and restart the app pool, and therefore it resulted in continuous, uninterrupted service. Conversely, it sounds like the behavior being designed for Asp.net Core is to stop, deploy, and start IIS. My question is, is there a difference in how requests will be served during the deployment period (short as it may be)? Is it possible that some Asp.net Core requests that coincide with deployment will see a "App is offline" page, whereas this wasn't the case before? (My deployment target is an Azure Web App)

[vincent1173]

@kirkone , Latest version of Azure App Service Manage Task Deployment is yet to be completed. Can you please try managing the app using Azure PowerShell Task.

You can start-stop the app service using the Azure PowerShell Task. Please use the following commands to manage app service.

Stop-AzureRmWebApp -ResourceGroupName "RG_Name" -Name "App_Service_Name"
Start-AzureRmWebApp -ResourceGroupName "RG_Name" -Name "App_Service_Name"

For start-stop Azure App Service Slot , we need to use different command.

Stop-AzureRmWebAppSlot -ResourceGroupName "RG_Name" -Name "App_Name" -Slot "Slot_Name"
Start-AzureRmWebAppSlot -ResourceGroupName "RG_Name" -Name "App_Name" -Slot "Slot_Name"

[vincent1173]

@ArashMotamedi ,The behavior of ASPCore and ASP.NET apps are different. There is an active thread with this issue. Will provide you the complete details, once the fix is planned.

[kirkone]

@vincentdass I will give this a try. so far I used the the Azure CLI task to do so but sometimes this failed with ResourceGroup not found error in my own build controllers. I think using the PS is the better option here. I did something similar here PublishWebPackage.ps1 but I tried to use only build in tasks. Thank you

[mvvsubbu]

There is another tracking issue, so I am closing this will update on that issue is resolved. You can find status here https://github.com/Microsoft/vsts-tasks/issues/3312

[thuru-zz]

In the new build definition there's an option to Rename the file, which sets the MSDEPLOY_RENAME_LOCKED_FILES = 1 in the app settings of the web app.

Also, the new build definition has the Take App Offiline checkbox.

Also, Azure app service has introduced a CI option (still in preview) this could solve the problems.

[hkusulja]

I have still the same problem, on Task - Deploy AzureRM App Service - v3, checked options: Take App Offline, Publish using Web Deploy, Rename locked files.

2017-08-01T14:11:53.9735840Z Info: Updating file (myProject\ServerBackEnd.exe). 2017-08-01T14:11:57.3279522Z ##[error]Failed to deploy web package to App Service. 2017-08-01T14:11:57.3279522Z ##[warning]Try to deploy app service again with Rename locked files option selected. 2017-08-01T14:11:57.3279522Z ##[error]Error Code: ERROR_FILE_IN_USE More Information: Web Deploy cannot modify the file 'ServerBackEnd.exe' on the destination because it is locked by an external process. In order to allow the publish operation to succeed, you may need to either restart your application to release the lock, or use the AppOffline rule handler for .Net applications on your next publish attempt. Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_FILE_IN_USE. Error count: 1.

2017-08-01T14:11:57.3279522Z ##[error]Error: C:\Program Files\IIS\Microsoft Web Deploy V3\msdeploy.exe failed with return code: 4294967295

[vincent1173]

@hkusulja , Did you try stopping the App Service, Deploy and start the same? You can use Azure App Service Manage Task to start/stop App Service.

Zero downtime deployment can also be achieved by deploying to a staging slot and swapping with production slot.

• Stop Staging slot
• Deploy to staging slot
• Start staging slot
• Swap with production

[hkusulja]

I am aware of that, I want to simplify the steps for smaller projects. I believe that option "Take App Offline" should be enough, to have a downtime and upload new app. So when will be fix in that direction?