search

OrchardCMS / Orchard

Orchard is a free, open source, community-focused Content Management System built on the ASP.NET MVC platform.
https://orchardproject.net
BSD 3-Clause "New" or "Revised" License
2.38k stars 1.12k forks source link

Instance randomly fails to start on Azure Cloud Service #5346

Open madmox opened 9 years ago

madmox commented 9 years ago

We deployed our solution on an Azure Cloud Service and activated the Azure auto-scaling functionality (2 instances from 8 A.M. to 19 P.M. during the week and 1 instance otherwise).

Orchard's version is 1.8.1, we have a few custom modules and themes and are using the multitenancy feature. Usually the deployment does not cause any trouble and the sites are working just fine.

But sometimes (~ once a week), when auto-scaling starts the 2nd instance (at 8 A.M. but not on any particular day during the week), the tenants fail to start on this instance. We then keep getting this error in the logs:

2015-03-18 07:14:27,046 [7] Orchard.Environment.DefaultOrchardHost - A tenant could not be started: <our tenant name>
Autofac.Core.DependencyResolutionException: None of the constructors found with 'Orchard.Environment.AutofacUtil.DynamicProxy2.ConstructorFinderWrapper' on type 'Orchard.Settings.CurrentSiteWorkContext' can be invoked with the available services and parameters:
Cannot resolve parameter 'Orchard.Settings.ISiteService siteService' of constructor 'Void .ctor(Orchard.Settings.ISiteService)'.
   at Autofac.Core.Activators.Reflection.ReflectionActivator.ActivateInstance(IComponentContext context, IEnumerable`1 parameters) in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Activators\Reflection\ReflectionActivator.cs:line 106
   at Autofac.Core.Resolving.InstanceLookup.Activate(IEnumerable`1 parameters) in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Resolving\InstanceLookup.cs:line 79
   at Autofac.Core.Lifetime.LifetimeScope.GetOrCreateAndShare(Guid id, Func`1 creator) in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Lifetime\LifetimeScope.cs:line 270
   at Autofac.Core.Resolving.InstanceLookup.Execute() in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Resolving\InstanceLookup.cs:line 64
   at Autofac.Core.Resolving.ResolveOperation.GetOrCreateInstance(ISharingLifetimeScope currentOperationScope, IComponentRegistration registration, IEnumerable`1 parameters) in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Resolving\ResolveOperation.cs:line 123
   at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Autofac.Features.Collections.CollectionRegistrationSource.<>c__DisplayClass4.<RegistrationsFor>b__0(IComponentContext c, IEnumerable`1 p) in c:\Projects\OSS\autofac\Core\Source\Autofac\Features\Collections\CollectionRegistrationSource.cs:line 75
   at Autofac.Core.Activators.Delegate.DelegateActivator.ActivateInstance(IComponentContext context, IEnumerable`1 parameters) in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Activators\Delegate\DelegateActivator.cs:line 69
   at Autofac.Core.Resolving.InstanceLookup.Activate(IEnumerable`1 parameters) in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Resolving\InstanceLookup.cs:line 79
   at Autofac.Core.Resolving.InstanceLookup.Execute() in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Resolving\InstanceLookup.cs:line 62
   at Autofac.Core.Resolving.ResolveOperation.GetOrCreateInstance(ISharingLifetimeScope currentOperationScope, IComponentRegistration registration, IEnumerable`1 parameters) in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Resolving\ResolveOperation.cs:line 123
   at Autofac.Core.Activators.Delegate.DelegateActivator.ActivateInstance(IComponentContext context, IEnumerable`1 parameters) in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Activators\Delegate\DelegateActivator.cs:line 69
   at Autofac.Core.Resolving.InstanceLookup.Activate(IEnumerable`1 parameters) in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Resolving\InstanceLookup.cs:line 79
   at Autofac.Core.Resolving.InstanceLookup.Execute() in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Resolving\InstanceLookup.cs:line 62
   at Autofac.Core.Resolving.ResolveOperation.GetOrCreateInstance(ISharingLifetimeScope currentOperationScope, IComponentRegistration registration, IEnumerable`1 parameters) in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Resolving\ResolveOperation.cs:line 123
   at Autofac.Core.Resolving.ResolveOperation.Execute(IComponentRegistration registration, IEnumerable`1 parameters) in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Resolving\ResolveOperation.cs:line 94
   at Autofac.Core.Lifetime.LifetimeScope.ResolveComponent(IComponentRegistration registration, IEnumerable`1 parameters) in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Lifetime\LifetimeScope.cs:line 232
   at Autofac.ResolutionExtensions.ResolveService(IComponentContext context, Service service, IEnumerable`1 parameters) in c:\Projects\OSS\autofac\Core\Source\Autofac\ResolutionExtensions.cs:line 346
   at Autofac.ResolutionExtensions.Resolve[TService](IComponentContext context, IEnumerable`1 parameters) in c:\Projects\OSS\autofac\Core\Source\Autofac\ResolutionExtensions.cs:line 197
   at Orchard.Environment.WorkContextModule.<Load>b__0(IComponentContext ctx) in c:\dev\orchard\src\Orchard\Environment\WorkContextModule.cs:line 19
   at Autofac.Builder.RegistrationBuilder.<>c__DisplayClass1`1.<ForDelegate>b__0(IComponentContext c, IEnumerable`1 p) in c:\Projects\OSS\autofac\Core\Source\Autofac\Builder\RegistrationBuilder.cs:line 36
   at Autofac.Core.Activators.Delegate.DelegateActivator.ActivateInstance(IComponentContext context, IEnumerable`1 parameters) in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Activators\Delegate\DelegateActivator.cs:line 69
   at Autofac.Core.Resolving.InstanceLookup.Activate(IEnumerable`1 parameters) in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Resolving\InstanceLookup.cs:line 79
   at Autofac.Core.Lifetime.LifetimeScope.GetOrCreateAndShare(Guid id, Func`1 creator) in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Lifetime\LifetimeScope.cs:line 270
   at Autofac.Core.Resolving.InstanceLookup.Execute() in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Resolving\InstanceLookup.cs:line 64
   at Autofac.Core.Resolving.ResolveOperation.GetOrCreateInstance(ISharingLifetimeScope currentOperationScope, IComponentRegistration registration, IEnumerable`1 parameters) in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Resolving\ResolveOperation.cs:line 123
   at Autofac.Core.Resolving.ResolveOperation.Execute(IComponentRegistration registration, IEnumerable`1 parameters) in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Resolving\ResolveOperation.cs:line 94
   at Autofac.Core.Lifetime.LifetimeScope.ResolveComponent(IComponentRegistration registration, IEnumerable`1 parameters) in c:\Projects\OSS\autofac\Core\Source\Autofac\Core\Lifetime\LifetimeScope.cs:line 232
   at Autofac.ResolutionExtensions.ResolveService(IComponentContext context, Service service, IEnumerable`1 parameters) in c:\Projects\OSS\autofac\Core\Source\Autofac\ResolutionExtensions.cs:line 346
   at Autofac.ResolutionExtensions.Resolve[TService](IComponentContext context, IEnumerable`1 parameters) in c:\Projects\OSS\autofac\Core\Source\Autofac\ResolutionExtensions.cs:line 197
   at Orchard.Environment.WorkContextAccessor.ThreadStaticScopeImplementation..ctor(IEnumerable`1 events, ILifetimeScope lifetimeScope, ConcurrentDictionary`2 contexts, Object workContextKey) in c:\dev\orchard\src\Orchard\Environment\WorkContextAccessor.cs:line 116
   at Orchard.Environment.WorkContextAccessor.CreateWorkContextScope() in c:\dev\orchard\src\Orchard\Environment\WorkContextAccessor.cs:line 66
   at Orchard.Environment.ShellBuilders.ShellContextFactory.CreateShellContext(ShellSettings settings) in c:\dev\orchard\src\Orchard\Environment\ShellBuilders\ShellContextFactory.cs:line 61
   at Orchard.Environment.DefaultOrchardHost.<CreateAndActivateShells>b__7(ShellSettings settings) in c:\dev\orchard\src\Orchard\Environment\DefaultOrchardHost.cs:line 136
; TraceSource 'w3wp.exe' event

Manually restarting the instance usually solves (until the next failing auto-scaling) the problem. Disabling auto-scaling reduces the impact of the issue (because instances are started less frequently), but it does not solve it completely: deployments regularly fail, and sometimes Azure restarts or moves deployments to another instance. Any idea what's going on?

Checkout the original discussion on codeplex: http://orchard.codeplex.com/discussions/587886

Piedone commented 9 years ago

We had the same symptons with Azure Web Apps, as elaborated in the linked discussion. With 1.9 such log messages were generated during deployment, but after the deployment finished the sites actually started fine. With 1.9 however the starts weren't starting any more and the error persisted.

The culprit was that the below line is needed in the Publish Profile:

<_DestinationType>AzureWebSite</_DestinationType>

in the existing <PropertyGroup>.

This is an Azure-specific config that is required as something is unique with Azure deployments.

I understand that yours is a Cloud Service but I suspect you should have a similar issue. Any way, your issue is surely related to how the instance is deployed, since you experience it during scaling.

jan-chvojka commented 9 years ago

I would like to join the fanclub of this issue. The thing is that Orchard is not runnig as Web App or Cloud Service but as Azure's VM (2008R2, IIS 7.5, Orchard 1.8.1). And it's not random but it happens after the deployment.

Application is deployed by VS Publish. Restart of the AppPool helps and after that it runs just fine. Is there anything I could check since it's a different environment?

Orchard.Environment.DefaultOrchardHost - (null) - A tenant could not be started: Default
Autofac.Core.DependencyResolutionException: None of the constructors found with 'Orchard.Environment.AutofacUtil.DynamicProxy2.ConstructorFinderWrapper' on type 'Orchard.Settings.CurrentSiteWorkContext' can be invoked with the available services and parameters:
Cannot resolve parameter 'Orchard.Settings.ISiteService siteService' of constructor 'Void .ctor(Orchard.Settings.ISiteService)'.
   at Autofac.Core.Activators.Reflection.ReflectionActivator.ActivateInstance(IComponentContext context, IEnumerable`1 parameters)
.
.
.
   at Orchard.Environment.ShellBuilders.ShellContextFactory.CreateShellContext(ShellSettings settings)
   at Orchard.Environment.DefaultOrchardHost.CreateShellContext(ShellSettings settings)
   at Orchard.Environment.DefaultOrchardHost.<CreateAndActivateShells>b__7(ShellSettings settings)
Piedone commented 9 years ago

@cwojke did you check whether you have any similar similar config issue than I mentioned?

jan-chvojka commented 9 years ago

Thanks for your reply. I checked my publishing profile but I didn't find anything unusual and because I'm not deploying it to AzureWebSite/CloudService I add nothing.

But as you mention it, I've found some example with <_DestinationType>AzureVirtualMachine... I will try it, but I'm not giving it much chance. It's an ordinary web deploy to VM (on Azure).

And up until last week, it worked fine. I will change log level and see if there isn't something else going on.