"An item with the same key has already been added."

[0xdeafcafe]

With automated git deployments set up, I'm receiving this error on deploy.

An item with the same key has already been added.

Full stack trace from azure logs below:

2015-03-01T16:38:11  Fetching changes.
2015-03-01T16:38:14  Updating submodules.
2015-03-01T16:38:14  Preparing deployment for commit id 'f322047390'.
2015-03-01T16:38:14  An item with the same key has already been added.
2015-03-01T16:38:14    An item with the same key has already been added.
2015-03-01T16:38:15        Error occurred, type: error, text: An item with the same key has already been added., stackTrace:    at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource)
   at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
   at System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value)
   at Kudu.Core.Deployment.Generator.SiteBuilderFactory.CreateBuilder(ITracer tracer, ILogger logger, IDeploymentSettingsManager settings, IFileFinder fileFinder)
   at Kudu.Core.Deployment.DeploymentManager.<Build>d__29.MoveNext()

[davidebbo]

A few questions to try to isolate:

• does this happen if you create a new site and connect the same repo? Or only on one specific site?
• what region is it in?
• can you share your site name, either directly or indirectly

[0xdeafcafe]

• happens only on this website, other websites with the same repo work fine.
• west europe
• yeah, it's baelor-api

[davidebbo]

Thank you, this will help investigate. And I assume that on that site, if you retry deploying, it happens consistently?

[0xdeafcafe]

It happens consistently. I added new commits (to change the commit id), and the same error is returned.

[davidebbo]

Is this an ASP.NET vNext project? We just deployed a change to make it work with the new beta 3, and if you're on Beta 1 or Beta 2, I think there could be issues. And as it stands, that new build is only deployed in West Europe so far, which is a strong sign.

Though the specific way that it's failing is puzzling. I think it's getting to this line, and blows up because that key already exist. @ahmelsayed any idea?

@0xdeafcafe any chance you can try moving to Beta 3?

[0xdeafcafe]

Yeah, it's vnext. I believe I'm on beta3. Here is my project.json

{
    "webroot": "wwwroot",
    "version": "1.0-bae-*",
    "authors": [ "Alex Forbes-Reed" ],
    "description": "Taylor Swift API - After all, she is bae.",
    "exclude": [
        "wwwroot",
        "node_modules",
        "bower_components"
    ],
    "bundleExclude": [
        "node_modules",
        "bower_components",
        "**.kproj",
        "**.user",
        "**.vspscc"
    ],
    "packExclude": [
        "**.kproj",
        "**.user",
        "**.vspscc"
    ],
    "dependencies": {
        "Microsoft.AspNet.Mvc": "6.0.0-beta3",
        "Microsoft.AspNet.Routing": "1.0.0-beta3",
        "Microsoft.AspNet.Server.IIS": "1.0.0-beta3",
        "Microsoft.AspNet.StaticFiles": "1.0.0-beta3",
        "Microsoft.AspNet.Diagnostics": "1.0.0-beta3",
        "Microsoft.AspNet.Diagnostics.Entity": "7.0.0-beta3",
        "Microsoft.AspNet.Mvc.WebApiCompatShim": "6.0.0-beta3",
        "Microsoft.Framework.ConfigurationModel": "1.0.0-beta3",
        "Microsoft.Framework.ConfigurationModel.Json": "1.0.0-beta3",
        "Microsoft.Framework.DependencyInjection": "1.0.0-beta3",

        "System.Security.Cryptography.Hashing": "4.0.0-beta",
        "System.Security.Cryptography.Hashing.Algorithms": "4.0.0-beta",
        "System.Security.Cryptography.Encoding": "4.0.0-beta",
        "System.Security.Cryptography.Encryption": "4.0.0-beta",
        "System.Security.Cryptography.DeriveBytes": "4.0.0-beta",
        "System.Security.Cryptography.Encryption.Aes": "4.0.0-beta",
        "System.Security.Cryptography.Encryption.RSA": "4.0.0-beta",
        "System.Security.Cryptography.X509Certificates": "4.0.0-beta",
        "System.Security.Cryptography.RandomNumberGenerator": "4.0.0-beta",

        "EntityFramework": "7.0.0-beta3",
        "EntityFramework.SqlServer": "7.0.0-beta3",
        "EntityFramework.Commands": "7.0.0-beta3"
    },
    "commands": {
        "ef": "EntityFramework.Commands"
    },
    "frameworks": {
        "aspnet50": { }
    },
    "compilationOptions": {
        "allowUnsafe": false,
        "warningsAsErrors": false,
        "languageVersion": "experimental"
    },
    "scripts": {
        "prepare": "echo during prepare",
        "prebuild": "echo before building",
        "postbuild": "echo after building",
        "prepack": "echo before packing",
        "postpack": "echo after packing",
        "prerestore": "echo before restoring packages",
        "postrestore": "echo after restoring packages"
    },
    "configurations": {
        "Debug": {
            "compilationOptions": {
                "define": [ "DEBUG", "TRACE" ]
            }
        },
        "Release": {
            "compilationOptions": {
                "define": [ "RELEASE", "TRACE" ],
                "optimize": true
            }
        }
    }
}

[davidebbo]

I see. So is this a new project? Note that beta3 just came out, and until yesterday, we had no support for it at all (see #1479). So it could be that the new logic has a bug, but was it working at all before today?

[0xdeafcafe]

It's not a new project, I've been running it for awhile. I added the kre version to the server through FTP and referenced the path in the web.config. I assume I can remove that now.

I remembered that I added this environment variable to the azure configuration page a few hours ago, and hadn't pushed since.

ASPNET_ENV=Release

After removing it, it deployed just fine. Do you know a proper way to get it to run on Release on the server? It defaults to Debug.

[davidebbo]

BTW, if you want to try going back to the previous build, you can set KUDU_EXTENSION_VERSION=43.40212.1385 in the Azure portal AppSetting (but remember to yank it later or you'll forever use this Kudu build!).

But based on your last comment, it could be that this issue existed even with the old build, but you had never tried with ASPNET_ENV=Release? If you are able to verify that, it would be great. i.e. go back to the old build AND use ASPNET_ENV=Release to see if that also blows up or not.

Anyway, I assume you're at least unblocked for now from deploying? @ahmelsayed should be able to comment further later today or tomorrow.

Sorry for the inconvenience!

[0xdeafcafe]

I pushed some more code - minimal changes - and it broke again with the same error - without the ASPNET_ENV environment variable. puzzling.

I set that environment variable you specified, and added ASPNET_ENV back, and this was the output in the logs

2015-03-01T17:53:55    KuduSync.NET from: 'D:\local\Temp\e6e920d2-c3de-443f-bf00-68eb222ffd86' to: 'D:\home\site'
2015-03-01T17:53:55    Copying file: 'ef.cmd'
2015-03-01T17:53:55    Copying file: 'approot\global.json'
2015-03-01T17:53:55    Deleting file: 'approot\packages\KRE-CLR-win-x86.1.0.0-beta3\bin\kre.clr.managed.dll'
2015-03-01T17:53:55    An error has occurred during web site deployment.
2015-03-01T17:53:55          Error occurred, type: error, text: Error: Access to the path 'kre.clr.managed.dll' is denied.
D:\Program Files (x86)\SiteExtensions\Kudu\43.40212.1385\bin\scripts\starter.cmd "D:\home\site\deployments\tools\deploy.cmd", stackTrace:    at Kudu.Core.Infrastructure.Executable.ExecuteInternal(ITracer tracer, Func`2 onWriteOutput, Func`2 onWriteError, Encoding encoding, String arguments, Object[] args)
   at Kudu.Core.Infrastructure.Executable.ExecuteWithProgressWriter(ILogger logger, ITracer tracer, String arguments, Object[] args)
   at Kudu.Core.Deployment.Generator.ExternalCommandBuilder.RunCommand(DeploymentContext context, String command)
2015-03-01T17:53:55  
2015-03-01T17:53:56        Error occurred, type: error, text: Error: Access to the path 'kre.clr.managed.dll' is denied.
D:\Program Files (x86)\SiteExtensions\Kudu\43.40212.1385\bin\scripts\starter.cmd "D:\home\site\deployments\tools\deploy.cmd", stackTrace:    at Kudu.Core.Infrastructure.Executable.ExecuteInternal(ITracer tracer, Func`2 onWriteOutput, Func`2 onWriteError, Encoding encoding, String arguments, Object[] args)
   at Kudu.Core.Infrastructure.Executable.ExecuteWithProgressWriter(ILogger logger, ITracer tracer, String arguments, Object[] args)
   at Kudu.Core.Deployment.Generator.ExternalCommandBuilder.RunCommand(DeploymentContext context, String command)
   at Kudu.Core.Deployment.Generator.GeneratorSiteBuilder.Build(DeploymentContext context)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
   at Kudu.Core.Deployment.DeploymentManager.<Build>d__29.MoveNext()

[davidebbo]

Note that instead of pushing new code, you can go to the deployments tab in the portal and Retry/Redeploy the current one.

So this is quite a different error, due to file in use. I would think that if you go back to the old Kudu and you don't set ASPNET_ENV, you should be back to exactly the same thing you had before this all started.

[ahmelsayed]

Yes, I didn't realize that IBuildPropertyProvider is a singleton. It doesn't affect git push scenario, but will break on continuous deployment. Sorry about that, I'll fix it.

[davidebbo]

Ok, so potential workaround for @0xdeafcafe is to restart the site before doing a deployment?

[ahmelsayed]

yes, I think that should work.

[0xdeafcafe]

Restarting before every deployment seems to be working perfectly. I'll keep doing that, thanks for your help guys, especially in such a timely manor. @ahmelsayed I don't supose you could give a rough ETA when your fix would be pushed to production?

[davidebbo]

Keeping open till we get the real fix in. ETA is tomorrow.

[0xdeafcafe]

thanks again guys 💃💃💃

[davidebbo]

Ok, the fix is now deployed in Production.

[0xdeafcafe]

Just pushed some code, deployment went fine. Thanks again for the speedy support!

[davidebbo]

Thanks for bringing up the issue. That allowed us to correct it before it went out too widely.