DbUp Task - 404 Error - nuget.exe not recognized error on *some* build servers

[sumo300]

We're getting a very weird error on only some of our build servers. I have attached a full task log, but a snippet of the error is below. Initially, I thought it was an issue of our group policy not allowing executables in the user profile temp folder, which is where nuget.exe is trying to execute for some reason in this task. This works fine on our primary build server, but we created 2 new ones and it fails on both.

2017-03-30T20:50:59.1038345Z ##[error]System.Net.WebException: The remote server returned an error: (404) Not Found.
2017-03-30T20:50:59.1038345Z    at Microsoft.PowerShell.Commands.WebRequestPSCmdlet.GetResponse(WebRequest request)
2017-03-30T20:50:59.1038345Z    at Microsoft.PowerShell.Commands.WebRequestPSCmdlet.ProcessRecord()
2017-03-30T20:50:59.2288418Z ##[error]System.Management.Automation.CommandNotFoundException: The term '.\nuget.exe' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

tasklog_4.zip

[johanclasson]

Hm...

The DbUp task looks for the dbup.dlls in the windows temp folder. If it can't find any, it fist downloads nuget.exe from nuget.org and next downloads the latest version of dbup through nuget.exe.

My guess is that your new build servers are not allowed to download nuget.exe. Hence the 404.

Can you try to download the dbup.dlls and place them in the windows temp folder yourself? https://github.com/johanclasson/vso-agent-tasks/tree/master/DbUpMigration#offline-scenarios

[johanclasson]

I made the task behave like this so that I would not have to create a new version of the task each time that DbUp mande a bugfix. If you have an other idea of how this could be done I am open for suggestions.

[sumo300]

NuGet is now something that is included in the agent and is actually in the path on these servers on top of that. However, where is nuget.exe being downloaded from. Our servers definitely have a proxy that require whitelisting. We have nuget.org whitelisted.

[johanclasson]

http://nuget.org/nuget.exe

Here is the line of code that downloads nuget.exe. https://github.com/johanclasson/vso-agent-tasks/blob/master/DbUpMigration/task/Update-DatabaseWithDbUp.ps1#L22

[sumo300]

I have an additional error log when I add nuget.exe to the folder where it is looking for it. Still fails. See attached. tasklog_5.zip

[johanclasson]

I just found out that one can use the PowerShell commandlet Get-Command to verify if an exe is available in the path. That would be something I could improve the task with. I will take a look on your logs now.

[sumo300]

I'd be willing to be a guinea pig for the update. We desperately need it as we have large releases going in parallel and need multiple build servers. I'm tackling the problem with our server folks, but if this improves the entire process and can find it properly rather than failing, it'd be great! And, I'd expect in other environments that are as restrictive as ours, there would be similar problems.

[johanclasson]

What I can see is that the first error is indeed because the download of nuget.exe fails.

2017-03-31T14:02:55.0826692Z ##[debug]GET http://nuget.org/nuget.exe with 0-byte payload
2017-03-31T14:02:55.1607901Z ##[error]System.Net.WebException: The remote server returned an error: (404) Not Found.

The task failed later on because that the DbUp-namespace could not be found due to that add-type could not find any dbup.dll. My guess is that nuget could not install (download) the dbup-package.

You can probably resolve all quickly by downloading the dbup.dll on your machine and copy it to the location %TEMP%\DatabaseMigration\dbup.*\lib\net35\DbUp.dll on the build servers.

[sumo300]

The dbup files are actually there. I saw that part of the error and checked the server.

Also, I logged onto one of the build servers failing and went to http://nuget.org/nuget.exe in a browser. It redirects to api.nuget.org for the actual download and sure enough, our proxy is seemingly blocking exes.

Ugh.

[johanclasson]

If you run Resolve-Path "$($env:TEMP)\DatabaseMigration\dbup.*\lib\net35\DbUp.dll" on one of the failing build server. Does it find anything?

[sumo300]

I'm not able to log in as the build user on the server, so running it wouldn't be accurate. But looking manually, I see this.

[sumo300]

To further follow up with this and a definitive answer as to what was happening here.

Some organizations enable a GPO that disables the download/execution of executables in the user profile temp folder (think C:\Users\[username]\AppData\temp). This GPO, if also applied to servers, which is the case in my organization, will cause the same issue. Once we got an exception added for our servers, all was well again.

Hopefully, my pull request #19 provides the ability for organizations who cannot disable this GPO to have nuget.exe installed in the path instead.

[johanclasson]

Thanks for the info! But I find it strange that the build server tried to download the dbup.dll even though that it existed in the temp directory already. I will try to merge #19 as soon as I have time. Probably this weekend.

[johanclasson]

I have now updated the extension in the marketplace to 0.10.7. Thanks again for your contribution!