search

umbraco / Umbraco-CMS

Umbraco is a free and open source .NET content management system helping you deliver delightful digital experiences.
https://umbraco.com
MIT License
4.48k stars 2.69k forks source link

Examine shut down and MainDom is released after SQL error #8612

Closed johnsonp57 closed 2 years ago

johnsonp57 commented 4 years ago

Unfortunately we have started getting similar symptoms re-occur on 3 of the sites which have the pre-release 8.6.4 dll's applied. It is not happening with such regularity, but is still an issue. Here are 2 stack traces from sites that we had to restart yesterday and today. Examine shuts down immediately after and MainDom is released.

image

image

Let us know if we can provide any further info to help diagnose this Thanks

Originally posted by @johnsonp57 in https://github.com/umbraco/Umbraco-CMS/issues/8215#issuecomment-670489311

Shazwazza commented 4 years ago

Thanks for submitting this info. Regarding this statement:

Examine shuts down immediately after and MainDom is released.

That is what is supposed to happen. Examine is controlled by MainDom. The previous issue is that MainDom was shutting down due to SQL errors instead of shutting down when the AppDomain shuts down. The purpose of MainDom is to synchronize access to resources between overlapping AppDomain instances. Part of the previous fix is to not shutdown MainDom due to unexpected errors. You can see here that we don't do that: https://github.com/umbraco/Umbraco-CMS/blob/v8/8.6/src/Umbraco.Core/Runtime/SqlMainDomLock.cs#L174

Are you able to explain the exact issue you are having? Is MainDom shutting down somehow before the AppDomain is shutting down or is this a different issue with similar symptoms? You can see in your logs when MainDom is shutting down/acquired/etc...

NikRimington commented 4 years ago

Hi @Shazwazza ,

(In reference to the same sites as above, so I believe this is all associated)

I think what we are seeing is AppDomains being triggered for no clear reason. Then, the site isn't actually shutting down fully as most of it is still available, in turn Examine has been released but the site is still up.

We've just had it again on a site and this is a snippet of the log files at the time we started getting examine issues.

{"@t":"2020-09-02T23:48:26.1046102Z","@mt":"Keep alive failed (at '{keepAlivePingUrl}').","@l":"Error","@x":"System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The remote name could not be resolved: 'sitedomain-website.azurewebsites.net'\r\n   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)\r\n   at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)\r\n   --- End of inner exception stack trace ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at Umbraco.Web.Scheduling.KeepAlive.<PerformRunAsync>d__5.MoveNext() in D:\\a\\1\\s\\src\\Umbraco.Web\\Scheduling\\KeepAlive.cs:line 69","keepAlivePingUrl":"https://sitedomain-website.azurewebsites.net/umbraco/api/keepalive/ping","SourceContext":"Umbraco.Web.Scheduling.KeepAlive","ProcessId":1884,"ProcessName":"w3wp","ThreadId":109,"AppDomainId":2,"AppDomainAppId":"LMW3SVC556649484ROOT","MachineName":"RD00155D02FFB8","Log4NetLevel":"ERROR"}
{"@t":"2020-09-02T23:48:31.6901740Z","@mt":"Stopping ({SignalSource})","SignalSource":"signal","SourceContext":"Umbraco.Core.Runtime.MainDom","ProcessId":1884,"ProcessName":"w3wp","ThreadId":58,"AppDomainId":2,"AppDomainAppId":"LMW3SVC556649484ROOT","MachineName":"RD00155D02FFB8","Log4NetLevel":"INFO "}
{"@t":"2020-09-02T23:48:33.5069202Z","@mt":"Server rd00155d02ffb8:1884:377e8f84 encountered an exception while sending heartbeat","@l":"Warning","@x":"System.Data.SqlClient.SqlException (0x80131904): A transport-level error has occurred when receiving results from the server. (provider: TCP Provider, error: 0 - The semaphore timeout period has expired.) ---> System.ComponentModel.Win32Exception (0x80004005): The semaphore timeout period has expired\r\n   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)\r\n   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)\r\n   at System.Data.SqlClient.TdsParserStateObject.ReadSniError(TdsParserStateObject stateObj, UInt32 error)\r\n   at System.Data.SqlClient.TdsParserStateObject.ReadSniSyncOverAsync()\r\n   at System.Data.SqlClient.TdsParserStateObject.TryReadNetworkPacket()\r\n   at System.Data.SqlClient.TdsParserStateObject.TryPrepareBuffer()\r\n   at System.Data.SqlClient.TdsParserStateObject.TryReadByte(Byte& value)\r\n   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)\r\n   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)\r\n   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)\r\n   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)\r\n   at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)\r\n   at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()\r\n   at Dapper.SqlMapper.ExecuteCommand(IDbConnection cnn, CommandDefinition& command, Action`2 paramReader)\r\n   at Dapper.SqlMapper.ExecuteImpl(IDbConnection cnn, CommandDefinition& command)\r\n   at Dapper.SqlMapper.Execute(IDbConnection cnn, String sql, Object param, IDbTransaction transaction, Nullable`1 commandTimeout, Nullable`1 commandType)\r\n   at Hangfire.SqlServer.SqlServerConnection.<>c__DisplayClass19_0.<Heartbeat>b__0(DbConnection connection)\r\n   at Hangfire.SqlServer.SqlServerStorage.<>c__DisplayClass32_0.<UseConnection>b__0(DbConnection connection)\r\n   at Hangfire.SqlServer.SqlServerStorage.UseConnection[T](DbConnection dedicatedConnection, Func`2 func)\r\n   at Hangfire.Server.ServerHeartbeatProcess.Execute(BackgroundProcessContext context)\r\nClientConnectionId:120e69f8-cd4a-4ad7-8aab-1ef990f10237\r\nError Number:121,State:0,Class:20\r\nClientConnectionId before routing:2e79c2ad-c0d7-4dc7-bde8-9b82b3631c2d\r\nRouting Destination:a319417a38e4.tr39.northeurope1-a.worker.database.windows.net,11111","SourceContext":"Hangfire.Server.ServerHeartbeatProcess","ProcessId":1884,"ProcessName":"w3wp","ThreadId":14,"AppDomainId":2,"AppDomainAppId":"LMW3SVC556649484ROOT","MachineName":"RD00155D02FFB8","Log4NetLevel":"WARN "}
{"@t":"2020-09-02T23:48:33.6788513Z","@mt":"Released from MainDom","SourceContext":"Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService","ProcessId":1884,"ProcessName":"w3wp","ThreadId":58,"AppDomainId":2,"AppDomainAppId":"LMW3SVC556649484ROOT","MachineName":"RD00155D02FFB8","Log4NetLevel":"INFO "}
{"@t":"2020-09-02T23:48:33.6944328Z","@mt":"{StartMessage} [Timing {TimingId}]","StartMessage":"Examine shutting down","TimingId":"824d981","SourceContext":"Umbraco.Web.Search.ExamineComponent","ProcessId":1884,"ProcessName":"w3wp","ThreadId":58,"AppDomainId":2,"AppDomainAppId":"LMW3SVC556649484ROOT","MachineName":"RD00155D02FFB8","Log4NetLevel":"INFO "}
{"@t":"2020-09-02T23:48:35.7195418Z","@mt":"{EndMessage} ({Duration}ms) [Timing {TimingId}]","EndMessage":"Completed.","Duration":2031,"TimingId":"824d981","SourceContext":"Umbraco.Web.Search.ExamineComponent","ProcessId":1884,"ProcessName":"w3wp","ThreadId":58,"AppDomainId":2,"AppDomainAppId":"LMW3SVC556649484ROOT","MachineName":"RD00155D02FFB8","Log4NetLevel":"INFO "}
{"@t":"2020-09-02T23:48:35.7820842Z","@mt":"Released ({SignalSource})","SignalSource":"signal","SourceContext":"Umbraco.Core.Runtime.MainDom","ProcessId":1884,"ProcessName":"w3wp","ThreadId":58,"AppDomainId":2,"AppDomainAppId":"LMW3SVC556649484ROOT","MachineName":"RD00155D02FFB8","Log4NetLevel":"INFO "}
{"@t":"2020-09-02T23:49:03.5239232Z","@mt":"Server rd00155d02ffb8:1884:377e8f84 is now able to continue sending heartbeats","SourceContext":"Hangfire.Server.ServerHeartbeatProcess","ProcessId":1884,"ProcessName":"w3wp","ThreadId":14,"AppDomainId":2,"AppDomainAppId":"LMW3SVC556649484ROOT","MachineName":"RD00155D02FFB8","Log4NetLevel":"INFO "}
{"@t":"2020-09-02T23:53:15.8964023Z","@mt":"Error getting Products - {{page}","@l":"Error","@x":"System.InvalidOperationException: Cannot create a Examine.LuceneEngine.Search.ISearchContext, the ExternalSearcher index either doesn't exist or the Examine.LuceneEngine.Providers.LuceneSearcher has been disposed\r\n   at Examine.LuceneEngine.Providers.LuceneSearcher.GetSearchContext() in C:\\projects\\examine-qvx04\\src\\Examine\\LuceneEngine\\Providers\\LuceneSearcher.cs:line 94\r\n   at Examine.LuceneEngine.Providers.BaseLuceneSearcher.CreateQuery(String category, BooleanOperation defaultOperation, Analyzer luceneAnalyzer, LuceneSearchOptions searchOptions) in C:\\projects\\examine-qvx04\\src\\Examine\\LuceneEngine\\Providers\\BaseLuceneSearcher.cs:line 64\r\n   at Examine.LuceneEngine.Providers.BaseLuceneSearcher.CreateQuery(String category, BooleanOperation defaultOperation) in C:\\projects\\examine-qvx04\\src\\Examine\\LuceneEngine\\Providers\\BaseLuceneSearcher.cs:line 49\r\n   at sitedomain.Core.Products.ProductsService.SearchForPosts(FilterModel filters, String sortOrder) in D:\\a\\1\\s\\src\\sitedomain.Core\\Products\\IProductsService.cs:line 59\r\n   at sitedomain.Core.Products.ProductsService.GetPagedPostsFeed(Int32 page, Int32 pageSize, FilterModel filters, String sortOrder) in D:\\a\\1\\s\\src\\sitedomain.Core\\Products\\IProductsService.cs:line 46","SourceContext":"sitedomain.Core.Products.ProductsService","ProcessId":1884,"ProcessName":"w3wp","ThreadId":42,"AppDomainId":2,"AppDomainAppId":"LMW3SVC556649484ROOT","MachineName":"RD00155D02FFB8","Log4NetLevel":"ERROR","HttpSessionId":"a1yau1qm1nzw2psj1xtng1hp","HttpRequestNumber":9,"HttpRequestId":"c042df3b-59d1-4039-87c7-20d3cf4ff584"}

I know that Pete thought that this was related to SQL timeouts, and they do occur around the time that we are seeing domain shutdowns being signalled.

As an FYI, on this site, I've just bumped the logging up to debug to try and capture more information as this has happened twice in a week for this site in-particular.

Shazwazza commented 4 years ago

Hi @NikRimington. When a background thread encounters an unexpected error, it will tear down the entire app domain. This is why it is critical to catch all unhandled exceptions that occur on background threads. Looking at your logs, the SQL error is occuring on a background thread since that is coming from Hangfire: Hangfire.Server.ServerHeartbeatProcess.Execute . There's potential that Hangfire isn't catching all unhandled exceptions on background threads and it's tearing down the app domain too. Might be worth investigating?

NikRimington commented 4 years ago

I'll have a look, but only one of the 3 sites having the issue is using Hangfire and background tasks. It just so happens to be the one I was able to get looks from for you.

Shazwazza commented 4 years ago

@NikRimington Do you have logs about startup and shutdown? Anytime the AppDomain is shutdown we log a detailed log about what actually shut the site down, whether it was a config change or something else. If you don't see that log anywhere then it means either your logging is excluding it, or the process has been forcefully terminated or torn down (i.e. background thread throwing unhandled exception). You can correspond shutdown and startup logs to see what is happening. In the log viewer you can search for "SourceContext='Umbraco.Web.UmbracoApplicationBase'"

and you'll see shutdown logs like:

image

But it sounds like what you are really saying is that MainDom has been triggered without the AppDomain being shutdown. You can view MainDom logs by searching: SourceContext='Umbraco.Core.Runtime.MainDom' and you'll get more info with Debug enabled. You can enable Debug logs specifically for MainDom by chaning your logs settins if you don't want to enable Debug globally. In serilog.config: <add key="serilog:minimum-level:override:Umbraco.Core.Runtime.MainDom" value="Debug" /> It will tell you what 'string' type has signaled MainDom:

image

There is a new known issue that may be affecting some of this strange behavior. I have a user already reporting that this has fixed one of their strange problems: https://github.com/umbraco/Umbraco-CMS/pull/8772 If you wish to try a build of that the artifacts are here https://dev.azure.com/umbraco/Umbraco%20Cms/_build/results?buildId=46849&view=artifacts&type=publishedArtifacts ... I really don't know if it's related though since there's nothing above that indicates such a thing.

Shazwazza commented 4 years ago

I'm unsure if this is related either but in the netcore project in integration tests it was discovered that the core itself isn't unbinding from static events. This was causing pinned stale examine instances to be bound to old stale events and trying to indexing when that manage instances was already disposed. I don't think this affects websites because when AppDomains shutdown it takes everything with them whereas in tests this is different but that said there's some probability that is causing strange behavior too https://github.com/umbraco/Umbraco-CMS/pull/8815

NikRimington commented 4 years ago

So, I've had a look through the logs I have at the minute but they don't have any reference to Umbraco.Web.UmbracoApplicationBase. But I think that could be because logging at that time was set to Warning. When the issue happened yesterday I've bumped the logging up to debug so that if it happens again we have more detailed logs (just have to wait for that.

With regards to your comment about exceptions in background threads, our custom code has try catches to catch exceptions and log them, then to handle them cleanly. In my testing, however, the only exception that has been thrown and, for some reason, not caught by our try catches (despite it appearing to be within a try), is coming from Umbraco's internal redirect module where its throwing a null exception when it can't find an existing item. This appears to be in an event handler somewhere, and is why I raised a different issue about raiseEvents: false not being adhered to in the SaveAndPublish method.

On a side note, I was looking to see how handle exceptions with Hangfire and came across this: https://docs.hangfire.io/en/latest/background-processing/dealing-with-exceptions.html, which indicates that they handle exceptions in a way to prevent the app shutting down (unless I'm miss understanding).

Hopefully the issue will crop up again in the next couple of days so I can get more detailed logs though.

NikRimington commented 4 years ago

Note: I'm hesitant with sharing the details in this comment as I'm not 100% sure they are related to the same issue although I think they could be, or at least partially.

Another site (not the one referenced in my previous comments), that does not use Hangfire, has had this issue. However it appears to have occurred directly after Azure has moved the site. The site was moved around 6am in the morning and shortly after there are some odd things in the logs followed by examine failures.

Log Details

```json {"@t":"2020-09-04T06:40:38.2839264Z","@mt":"{StartMessage} [Timing {TimingId}]","StartMessage":"Booting Umbraco 8.6.4.","TimingId":"972d694","SourceContext":"Umbraco.Core.Runtime.CoreRuntime","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:38.3151757Z","@mt":"Booting site '{HostingSiteName}', app '{HostingApplicationID}', path '{HostingPhysicalPath}', server '{MachineName}'.","HostingSiteName":"RealDomain-website__39cd","HostingApplicationID":"/LM/W3SVC/1572358820/ROOT","HostingPhysicalPath":"D:\\home\\site\\wwwroot\\","MachineName":"RD281878C9A228","SourceContext":"Umbraco.Core.Runtime.CoreRuntime","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:39.2370503Z","@mt":"Acquiring.","SourceContext":"Umbraco.Core.Runtime.MainDom","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:42.3307930Z","@mt":"Acquired.","SourceContext":"Umbraco.Core.Runtime.MainDom","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:42.7526688Z","@mt":"{StartMessage} [Timing {TimingId}]","StartMessage":"Resolving composer types.","TimingId":"0f980b6","SourceContext":"Umbraco.Core.Runtime.CoreRuntime","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:43.6589186Z","@mt":"{EndMessage} ({Duration}ms) [Timing {TimingId}]","EndMessage":"Resolved.","Duration":911,"TimingId":"0f980b6","SourceContext":"Umbraco.Core.Runtime.CoreRuntime","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:50.7057807Z","@mt":"Registered with MainDom, localContentDbExists? {LocalContentDbExists}, localMediaDbExists? {LocalMediaDbExists}","LocalContentDbExists":false,"LocalMediaDbExists":false,"SourceContext":"Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:50.7057807Z","@mt":"Creating the content store, localContentDbExists? {LocalContentDbExists}","LocalContentDbExists":false,"SourceContext":"Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:50.7057807Z","@mt":"Creating the media store, localMediaDbExists? {LocalMediaDbExists}","LocalMediaDbExists":false,"SourceContext":"Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:52.8464162Z","@mt":"{StartMessage} [Timing {TimingId}]","StartMessage":"Loading content from database","TimingId":"3cffdd1","SourceContext":"Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:53.4557797Z","@mt":"{EndMessage} ({Duration}ms) [Timing {TimingId}]","EndMessage":"Completed.","Duration":603,"TimingId":"3cffdd1","SourceContext":"Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:54.0182811Z","@mt":"{StartMessage} [Timing {TimingId}]","StartMessage":"Loading media from database","TimingId":"6b551a8","SourceContext":"Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:54.7839013Z","@mt":"{EndMessage} ({Duration}ms) [Timing {TimingId}]","EndMessage":"Completed.","Duration":770,"TimingId":"6b551a8","SourceContext":"Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:55.6589051Z","@mt":"Profiler is VoidProfiler, not profiling (must run debug mode to profile).","SourceContext":"Umbraco.Web.Logging.WebProfilerComponent","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:56.1745237Z","@mt":"Initializing Umbraco internal event handlers for cache refreshing.","SourceContext":"Umbraco.Web.Cache.DistributedCacheBinderComponent","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:56.2995242Z","@mt":"Starting '{MigrationName}'...","MigrationName":"UmbracoForms","SourceContext":"Umbraco.Core.Migrations.MigrationPlan","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:56.2995242Z","@mt":"At {OrigState}","OrigState":"{forms-init-complete}","SourceContext":"Umbraco.Core.Migrations.MigrationPlan","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:56.2995242Z","@mt":"Done (pending scope completion).","SourceContext":"Umbraco.Core.Migrations.MigrationPlan","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:56.3620248Z","@mt":"Initializing Umbraco Forms Events to trigger Cache Refreshers","SourceContext":"Umbraco.Forms.Core.Components.UmbracoFormsComposer","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:56.3932743Z","@mt":"Adding examine event handlers for {RegisteredIndexers} index providers.","RegisteredIndexers":3,"SourceContext":"Umbraco.Web.Search.ExamineComponent","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:56.4401517Z","@mt":"No last synced Id found, this generally means this is a new server/install. The server will build its caches and indexes, and then adjust its last synced Id to the latest found in the database and maintain cache updates based on that Id.","@l":"Warning","SourceContext":"Umbraco.Core.Sync.DatabaseServerMessenger","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"WARN ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:56.5026492Z","@mt":"{StartMessage} [Timing {TimingId}]","StartMessage":"Loading content from database","TimingId":"81e1fe6","SourceContext":"Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:56.8723349Z","@mt":"{EndMessage} ({Duration}ms) [Timing {TimingId}]","EndMessage":"Completed.","Duration":368,"TimingId":"81e1fe6","SourceContext":"Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:56.9035841Z","@mt":"{StartMessage} [Timing {TimingId}]","StartMessage":"Loading media from database","TimingId":"9ee1f60","SourceContext":"Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:57.2368945Z","@mt":"{EndMessage} ({Duration}ms) [Timing {TimingId}]","EndMessage":"Completed.","Duration":331,"TimingId":"9ee1f60","SourceContext":"Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:57.2368945Z","@mt":"Starting initialize async background thread.","SourceContext":"Umbraco.Web.Search.BackgroundIndexRebuilder","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:59.5442075Z","@mt":"uSync: Initializing events for {count} Handlers","count":8,"SourceContext":"uSync8.BackOffice.uSyncBackofficeComponent","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:59.5598278Z","@mt":"uSync: Startup Processes Complete {ElapsedMilliseconds}ms","ElapsedMilliseconds":205,"SourceContext":"uSync8.BackOffice.uSyncBackofficeComponent","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:59.5910774Z","@mt":"Forcing index {IndexerName} to be unlocked since it was left in a locked state","IndexerName":"ExternalIndex","SourceContext":"Umbraco.Examine.ExamineExtensions","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:59.5910774Z","@mt":"Call was made to RebuildIndexes but the task runner for rebuilding is already running","@l":"Warning","SourceContext":"Umbraco.Web.Search.BackgroundIndexRebuilder","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"WARN ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:40:59.5910774Z","@mt":"{EndMessage} ({Duration}ms) [Timing {TimingId}]","EndMessage":"Booted.","Duration":21329,"TimingId":"972d694","SourceContext":"Umbraco.Core.Runtime.CoreRuntime","ProcessId":4576,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"19ff438d-43de-4333-81a9-b172d0183927"} {"@t":"2020-09-04T06:41:00.5754525Z","@mt":"New url {Url} detected, re-discovering application url.","Url":"https://localhost/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":4576,"ProcessName":"w3wp","ThreadId":7,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":2,"HttpRequestId":"5667f0a5-2b52-4203-81b3-526bdf599599"} {"@t":"2020-09-04T06:41:00.5754525Z","@mt":"ApplicationUrl: {UmbracoAppUrl} (using web.routing/@umbracoApplicationUrl)","UmbracoAppUrl":"https://www.RealDomain.co.uk/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":4576,"ProcessName":"w3wp","ThreadId":7,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":2,"HttpRequestId":"5667f0a5-2b52-4203-81b3-526bdf599599"} {"@t":"2020-09-04T06:43:15.8065210Z","@mt":"Error indexing queue items","@l":"Error","@x":"System.NullReferenceException: Object reference not set to an instance of an object.\r\n at RealDomain.Core.ExamineStartup.RegisterExamineEventsComponent.AddIndexesForCategoryTags[T](IndexingItemEventArgs e, Int32 contentId) in D:\\a\\1\\s\\src\\RealDomain.Core\\ExamineStartup\\RegisterExamineEventsComponent.cs:line 67\r\n at RealDomain.Core.ExamineStartup.RegisterExamineEventsComponent.RegisterExamineEventsComponent_TransformingIndexValues(Object sender, IndexingItemEventArgs e) in D:\\a\\1\\s\\src\\RealDomain.Core\\ExamineStartup\\RegisterExamineEventsComponent.cs:line 106\r\n at Umbraco.Examine.UmbracoExamineIndex.OnTransformingIndexValues(IndexingItemEventArgs e) in D:\\a\\1\\s\\src\\Umbraco.Examine\\UmbracoExamineIndex.cs:line 173\r\n at Examine.LuceneEngine.Providers.LuceneIndex.ProcessIndexQueueItem(IndexOperation op, IndexWriter writer) in C:\\projects\\examine-qvx04\\src\\Examine\\LuceneEngine\\Providers\\LuceneIndex.cs:line 1251\r\n at Examine.LuceneEngine.Providers.LuceneIndex.ProcessQueueItem(IndexOperation item, IndexWriter writer) in C:\\projects\\examine-qvx04\\src\\Examine\\LuceneEngine\\Providers\\LuceneIndex.cs:line 1067\r\n at Examine.LuceneEngine.Providers.LuceneIndex.ForceProcessQueueItems(Boolean block) in C:\\projects\\examine-qvx04\\src\\Examine\\LuceneEngine\\Providers\\LuceneIndex.cs:line 913","SourceContext":"Umbraco.Examine.UmbracoContentIndex","ProcessId":4576,"ProcessName":"w3wp","ThreadId":20,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"ERROR"} {"@t":"2020-09-04T06:43:17.9940205Z","@mt":"Error indexing queue items","@l":"Error","@x":"System.NullReferenceException: Object reference not set to an instance of an object.\r\n at RealDomain.Core.ExamineStartup.RegisterExamineEventsComponent.AddIndexesForCategoryTags[T](IndexingItemEventArgs e, Int32 contentId) in D:\\a\\1\\s\\src\\RealDomain.Core\\ExamineStartup\\RegisterExamineEventsComponent.cs:line 67\r\n at RealDomain.Core.ExamineStartup.RegisterExamineEventsComponent.RegisterExamineEventsComponent_TransformingIndexValues(Object sender, IndexingItemEventArgs e) in D:\\a\\1\\s\\src\\RealDomain.Core\\ExamineStartup\\RegisterExamineEventsComponent.cs:line 106\r\n at Umbraco.Examine.UmbracoExamineIndex.OnTransformingIndexValues(IndexingItemEventArgs e) in D:\\a\\1\\s\\src\\Umbraco.Examine\\UmbracoExamineIndex.cs:line 173\r\n at Examine.LuceneEngine.Providers.LuceneIndex.ProcessIndexQueueItem(IndexOperation op, IndexWriter writer) in C:\\projects\\examine-qvx04\\src\\Examine\\LuceneEngine\\Providers\\LuceneIndex.cs:line 1251\r\n at Examine.LuceneEngine.Providers.LuceneIndex.ProcessQueueItem(IndexOperation item, IndexWriter writer) in C:\\projects\\examine-qvx04\\src\\Examine\\LuceneEngine\\Providers\\LuceneIndex.cs:line 1067\r\n at Examine.LuceneEngine.Providers.LuceneIndex.ForceProcessQueueItems(Boolean block) in C:\\projects\\examine-qvx04\\src\\Examine\\LuceneEngine\\Providers\\LuceneIndex.cs:line 913","SourceContext":"Umbraco.Examine.UmbracoContentIndex","ProcessId":4576,"ProcessName":"w3wp","ThreadId":20,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"ERROR"} {"@t":"2020-09-04T06:43:36.3999883Z","@mt":"New url {Url} detected, re-discovering application url.","Url":"https://www.RealDomain.us/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":4576,"ProcessName":"w3wp","ThreadId":7,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":3,"HttpRequestId":"1c6b33be-9172-4620-8e33-22aabf5b4baa"} {"@t":"2020-09-04T06:43:36.3999883Z","@mt":"ApplicationUrl: {UmbracoAppUrl} (using web.routing/@umbracoApplicationUrl)","UmbracoAppUrl":"https://www.RealDomain.co.uk/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":4576,"ProcessName":"w3wp","ThreadId":7,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":3,"HttpRequestId":"1c6b33be-9172-4620-8e33-22aabf5b4baa"} {"@t":"2020-09-04T06:43:36.4156143Z","@mt":"New url {Url} detected, re-discovering application url.","Url":"https://www.RealDomain.co.uk:443/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":4576,"ProcessName":"w3wp","ThreadId":12,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":4,"HttpRequestId":"86886e20-a95e-44f3-b048-f0cf163cff3b"} {"@t":"2020-09-04T06:43:36.4156143Z","@mt":"ApplicationUrl: {UmbracoAppUrl} (using web.routing/@umbracoApplicationUrl)","UmbracoAppUrl":"https://www.RealDomain.co.uk/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":4576,"ProcessName":"w3wp","ThreadId":12,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":4,"HttpRequestId":"86886e20-a95e-44f3-b048-f0cf163cff3b"} {"@t":"2020-09-04T06:43:43.8685992Z","@mt":"New url {Url} detected, re-discovering application url.","Url":"https://www.RealDomain.us:443/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":4576,"ProcessName":"w3wp","ThreadId":17,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":5,"HttpRequestId":"c4d4ac97-767d-48e6-b9cd-7bd534ac84ca"} {"@t":"2020-09-04T06:43:43.8685992Z","@mt":"ApplicationUrl: {UmbracoAppUrl} (using web.routing/@umbracoApplicationUrl)","UmbracoAppUrl":"https://www.RealDomain.co.uk/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":4576,"ProcessName":"w3wp","ThreadId":17,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":5,"HttpRequestId":"c4d4ac97-767d-48e6-b9cd-7bd534ac84ca"} {"@t":"2020-09-04T06:54:01.1696217Z","@mt":"Exception","@l":"Error","@x":"ImageProcessor.Common.Exceptions.ImageProcessingException: GetResponseAsync 132 : Response status code does not indicate success: 404 (The specified blob does not exist.).","SourceContext":"ImageProcessor.Web.Helpers.RemoteFile","ProcessId":4576,"ProcessName":"w3wp","ThreadId":57,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"ERROR"} {"@t":"2020-09-04T06:54:01.1696217Z","@mt":"Exception","@l":"Error","@x":"ImageProcessor.Common.Exceptions.ImageProcessingException: ProcessImageAsync 609 : No image exists at https://RealDomaincouk.blob.core.windows.net/media/2565/cr0015_lowres.jpg","SourceContext":"ImageProcessor.Web.HttpModules.ImageProcessingModule","ProcessId":4576,"ProcessName":"w3wp","ThreadId":57,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"ERROR"} ```

The thing that stands out is towards the bottom of this segment of logs where the site goes through discovering a load of different application URLS. It starts of getting localhost, then a US domain, then a UK domain, then US and finally back to UK. This seems odd to me as there is a specific domain set in UmbracoSettings, which is the UK domain.

I've cut the log snippet short, before the examine indexes start showing just for ease.

Additional logs from before Azure changed machine names

```json {"@t":"2020-09-04T01:22:44.6945382Z","@mt":"Error executing child request for handler 'System.Web.Mvc.HttpHandlerUtil+ServerExecuteHttpHandlerAsyncWrapper'.","SourceContext":"ASP._Page_Views_Partials_flexiblecontent__latestNews_cshtml","ProcessId":4000,"ProcessName":"w3wp","ThreadId":55,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"2253sx4foor1ic4dwzcgvst0","HttpRequestNumber":9,"HttpRequestId":"b0c500d4-d137-4ab1-b08a-860d2a19b89d"} {"@t":"2020-09-04T02:09:29.6935711Z","@mt":"{StartMessage} [Timing {TimingId}]","StartMessage":"Executing Umbraco RouteDefinition controller","TimingId":"65a8172","SourceContext":"Umbraco.Web.Mvc.UmbracoPageResult","ProcessId":4000,"ProcessName":"w3wp","ThreadId":21,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"degw2hxzhhvc5ykhbqh1fnr0","HttpRequestNumber":10,"HttpRequestId":"223e90f6-dc1d-4d4d-ae3a-80194a85b3de"} {"@t":"2020-09-04T02:09:29.8341963Z","@mt":"{EndMessage} ({Duration}ms) [Timing {TimingId}]","EndMessage":"Finished","Duration":152,"TimingId":"65a8172","SourceContext":"Umbraco.Web.Mvc.UmbracoPageResult","ProcessId":4000,"ProcessName":"w3wp","ThreadId":21,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"degw2hxzhhvc5ykhbqh1fnr0","HttpRequestNumber":10,"HttpRequestId":"223e90f6-dc1d-4d4d-ae3a-80194a85b3de"} {"@t":"2020-09-04T02:51:27.5560709Z","@mt":"New url {Url} detected, re-discovering application url.","Url":"https://RealDomain-website.azurewebsites.net:443/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":4000,"ProcessName":"w3wp","ThreadId":29,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":11,"HttpRequestId":"80a3147c-a54a-4f4d-ad7a-51cb190d7d8e"} {"@t":"2020-09-04T02:51:27.5560709Z","@mt":"ApplicationUrl: {UmbracoAppUrl} (using web.routing/@umbracoApplicationUrl)","UmbracoAppUrl":"https://www.RealDomain.co.uk/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":4000,"ProcessName":"w3wp","ThreadId":29,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":11,"HttpRequestId":"80a3147c-a54a-4f4d-ad7a-51cb190d7d8e"} {"@t":"2020-09-04T03:19:46.0165245Z","@mt":"{StartMessage} [Timing {TimingId}]","StartMessage":"Booting Umbraco 8.6.4.","TimingId":"a5577fb","SourceContext":"Umbraco.Core.Runtime.CoreRuntime","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:19:46.0790255Z","@mt":"Booting site '{HostingSiteName}', app '{HostingApplicationID}', path '{HostingPhysicalPath}', server '{MachineName}'.","HostingSiteName":"RealDomain-website__39cd","HostingApplicationID":"/LM/W3SVC/1572358820/ROOT","HostingPhysicalPath":"D:\\home\\site\\wwwroot\\","MachineName":"RD281878C76798","SourceContext":"Umbraco.Core.Runtime.CoreRuntime","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:19:46.6571566Z","@mt":"Acquiring.","SourceContext":"Umbraco.Core.Runtime.MainDom","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:19:49.0634268Z","@mt":"Acquired.","SourceContext":"Umbraco.Core.Runtime.MainDom","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:19:49.5321842Z","@mt":"{StartMessage} [Timing {TimingId}]","StartMessage":"Resolving composer types.","TimingId":"81d38bf","SourceContext":"Umbraco.Core.Runtime.CoreRuntime","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:19:52.5570368Z","@mt":"{EndMessage} ({Duration}ms) [Timing {TimingId}]","EndMessage":"Resolved.","Duration":3021,"TimingId":"81d38bf","SourceContext":"Umbraco.Core.Runtime.CoreRuntime","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:19:57.4008860Z","@mt":"Registered with MainDom, localContentDbExists? {LocalContentDbExists}, localMediaDbExists? {LocalMediaDbExists}","LocalContentDbExists":true,"LocalMediaDbExists":true,"SourceContext":"Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:19:57.4008860Z","@mt":"Creating the content store, localContentDbExists? {LocalContentDbExists}","LocalContentDbExists":true,"SourceContext":"Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:19:57.4008860Z","@mt":"Creating the media store, localMediaDbExists? {LocalMediaDbExists}","LocalMediaDbExists":true,"SourceContext":"Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:19:58.4790085Z","@mt":"{StartMessage} [Timing {TimingId}]","StartMessage":"Loading content from local cache file","TimingId":"98e587c","SourceContext":"Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:19:58.6352596Z","@mt":"{EndMessage} ({Duration}ms) [Timing {TimingId}]","EndMessage":"Completed.","Duration":167,"TimingId":"98e587c","SourceContext":"Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:19:58.6977549Z","@mt":"{StartMessage} [Timing {TimingId}]","StartMessage":"Loading media from local cache file","TimingId":"ecc4a91","SourceContext":"Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:19:58.8071291Z","@mt":"{EndMessage} ({Duration}ms) [Timing {TimingId}]","EndMessage":"Completed.","Duration":107,"TimingId":"ecc4a91","SourceContext":"Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:19:59.2446316Z","@mt":"Profiler is VoidProfiler, not profiling (must run debug mode to profile).","SourceContext":"Umbraco.Web.Logging.WebProfilerComponent","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:20:00.3227517Z","@mt":"Initializing Umbraco internal event handlers for cache refreshing.","SourceContext":"Umbraco.Web.Cache.DistributedCacheBinderComponent","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:20:00.6508764Z","@mt":"Adding examine event handlers for {RegisteredIndexers} index providers.","RegisteredIndexers":3,"SourceContext":"Umbraco.Web.Search.ExamineComponent","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:20:00.9789984Z","@mt":"Starting '{MigrationName}'...","MigrationName":"UmbracoForms","SourceContext":"Umbraco.Core.Migrations.MigrationPlan","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:20:00.9789984Z","@mt":"At {OrigState}","OrigState":"{forms-init-complete}","SourceContext":"Umbraco.Core.Migrations.MigrationPlan","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:20:00.9946237Z","@mt":"Done (pending scope completion).","SourceContext":"Umbraco.Core.Migrations.MigrationPlan","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:20:01.0258739Z","@mt":"Forcing index {IndexerName} to be unlocked since it was left in a locked state","IndexerName":"UmbracoFormsRecordsIndex","SourceContext":"Umbraco.Forms.Core.Components.UmbracoFormsComponent","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:20:01.0414966Z","@mt":"Initializing Umbraco Forms Events to trigger Cache Refreshers","SourceContext":"Umbraco.Forms.Core.Components.UmbracoFormsComposer","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:20:01.1352501Z","@mt":"uSync: Initializing events for {count} Handlers","count":8,"SourceContext":"uSync8.BackOffice.uSyncBackofficeComponent","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:20:01.1508751Z","@mt":"uSync: Startup Processes Complete {ElapsedMilliseconds}ms","ElapsedMilliseconds":104,"SourceContext":"uSync8.BackOffice.uSyncBackofficeComponent","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:20:01.1977476Z","@mt":"Forcing index {IndexerName} to be unlocked since it was left in a locked state","IndexerName":"ExternalIndex","SourceContext":"Umbraco.Examine.ExamineExtensions","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:20:01.2133754Z","@mt":"Starting initialize async background thread.","SourceContext":"Umbraco.Web.Search.BackgroundIndexRebuilder","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:20:01.2290019Z","@mt":"{EndMessage} ({Duration}ms) [Timing {TimingId}]","EndMessage":"Booted.","Duration":15225,"TimingId":"a5577fb","SourceContext":"Umbraco.Core.Runtime.CoreRuntime","ProcessId":9840,"ProcessName":"w3wp","ThreadId":1,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":1,"HttpRequestId":"4d166e22-4355-44e7-8d38-451bc673b9f8"} {"@t":"2020-09-04T03:20:03.0727436Z","@mt":"New url {Url} detected, re-discovering application url.","Url":"https://localhost/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":9840,"ProcessName":"w3wp","ThreadId":10,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":2,"HttpRequestId":"686cf359-1cf5-480c-b010-e7ce5c4dba5a"} {"@t":"2020-09-04T03:20:03.0727436Z","@mt":"ApplicationUrl: {UmbracoAppUrl} (using web.routing/@umbracoApplicationUrl)","UmbracoAppUrl":"https://www.RealDomain.co.uk/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":9840,"ProcessName":"w3wp","ThreadId":10,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":2,"HttpRequestId":"686cf359-1cf5-480c-b010-e7ce5c4dba5a"} {"@t":"2020-09-04T03:20:10.3539620Z","@mt":"New url {Url} detected, re-discovering application url.","Url":"https://www.RealDomain.co.uk:443/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":9840,"ProcessName":"w3wp","ThreadId":12,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":3,"HttpRequestId":"07e017d8-35fe-43cf-ac7d-99a8e0e9aa75"} {"@t":"2020-09-04T03:20:10.3539620Z","@mt":"ApplicationUrl: {UmbracoAppUrl} (using web.routing/@umbracoApplicationUrl)","UmbracoAppUrl":"https://www.RealDomain.co.uk/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":9840,"ProcessName":"w3wp","ThreadId":12,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":3,"HttpRequestId":"07e017d8-35fe-43cf-ac7d-99a8e0e9aa75"} {"@t":"2020-09-04T03:20:11.0102054Z","@mt":"New url {Url} detected, re-discovering application url.","Url":"https://RealDomain-website.azurewebsites.net:443/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":9840,"ProcessName":"w3wp","ThreadId":10,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":4,"HttpRequestId":"66ba328e-7c5c-4d96-94c6-0d9d96c93bf3"} {"@t":"2020-09-04T03:20:11.0102054Z","@mt":"ApplicationUrl: {UmbracoAppUrl} (using web.routing/@umbracoApplicationUrl)","UmbracoAppUrl":"https://www.RealDomain.co.uk/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":9840,"ProcessName":"w3wp","ThreadId":10,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":4,"HttpRequestId":"66ba328e-7c5c-4d96-94c6-0d9d96c93bf3"} {"@t":"2020-09-04T03:20:11.8227032Z","@mt":"New url {Url} detected, re-discovering application url.","Url":"https://www.RealDomain.us:443/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":9840,"ProcessName":"w3wp","ThreadId":6,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":5,"HttpRequestId":"206ad4a0-fd14-49f2-8dbe-b913a9fde810"} {"@t":"2020-09-04T03:20:11.8227032Z","@mt":"ApplicationUrl: {UmbracoAppUrl} (using web.routing/@umbracoApplicationUrl)","UmbracoAppUrl":"https://www.RealDomain.co.uk/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":9840,"ProcessName":"w3wp","ThreadId":6,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":5,"HttpRequestId":"206ad4a0-fd14-49f2-8dbe-b913a9fde810"} {"@t":"2020-09-04T03:20:18.6038144Z","@mt":"New url {Url} detected, re-discovering application url.","Url":"https://www.RealDomain.us/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":9840,"ProcessName":"w3wp","ThreadId":12,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":6,"HttpRequestId":"2d231f15-c6fa-4fa7-bd90-51ad0de031c0"} {"@t":"2020-09-04T03:20:18.6038144Z","@mt":"ApplicationUrl: {UmbracoAppUrl} (using web.routing/@umbracoApplicationUrl)","UmbracoAppUrl":"https://www.RealDomain.co.uk/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":9840,"ProcessName":"w3wp","ThreadId":12,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpRequestNumber":6,"HttpRequestId":"2d231f15-c6fa-4fa7-bd90-51ad0de031c0"} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Scheduled health check results:","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Checks for '{HealthCheckName}' all completed successfully.","HealthCheckName":"Folder & File Permissions","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Result for {HealthCheckName}: {HealthCheckResult}, Message: '{HealthCheckMessage}'","HealthCheckName":"Folder & File Permissions","HealthCheckResult":"Success","HealthCheckMessage":"All folders have the correct permissions set.","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Result for {HealthCheckName}: {HealthCheckResult}, Message: '{HealthCheckMessage}'","HealthCheckName":"Folder & File Permissions","HealthCheckResult":"Success","HealthCheckMessage":"All files have the correct permissions set.","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Checks for '{HealthCheckName}' all completed successfully.","HealthCheckName":"SMTP Settings","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Result for {HealthCheckName}: {HealthCheckResult}, Message: '{HealthCheckMessage}'","HealthCheckName":"SMTP Settings","HealthCheckResult":"Success","HealthCheckMessage":"SMTP settings are configured correctly and the service is operating as expected.","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Checks for '{HealthCheckName}' all completed successfully.","HealthCheckName":"Macro errors","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Result for {HealthCheckName}: {HealthCheckResult}, Message: '{HealthCheckMessage}'","HealthCheckName":"Macro errors","HealthCheckResult":"Success","HealthCheckMessage":"MacroErrors are set to 'inline'.","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Checks for '{HealthCheckName}' all completed successfully.","HealthCheckName":"Notification Email Settings","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Result for {HealthCheckName}: {HealthCheckResult}, Message: '{HealthCheckMessage}'","HealthCheckName":"Notification Email Settings","HealthCheckResult":"Success","HealthCheckMessage":"Notification email has been set to no-reply@RealDomain.co.uk.","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Checks for '{HealthCheckName}' all completed successfully.","HealthCheckName":"Try Skip IIS Custom Errors","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Result for {HealthCheckName}: {HealthCheckResult}, Message: '{HealthCheckMessage}'","HealthCheckName":"Try Skip IIS Custom Errors","HealthCheckResult":"Success","HealthCheckMessage":"Try Skip IIS Custom Errors is set to 'true' and you're using IIS version '10.0'.","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Checks for '{HealthCheckName}' all completed successfully.","HealthCheckName":"Custom Errors","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Result for {HealthCheckName}: {HealthCheckResult}, Message: '{HealthCheckMessage}'","HealthCheckName":"Custom Errors","HealthCheckResult":"Success","HealthCheckMessage":"Custom errors are set to 'On'.","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Checks for '{HealthCheckName}' all completed successfully.","HealthCheckName":"Trace Mode","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Result for {HealthCheckName}: {HealthCheckResult}, Message: '{HealthCheckMessage}'","HealthCheckName":"Trace Mode","HealthCheckResult":"Success","HealthCheckMessage":"Trace mode is disabled.","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Checks for '{HealthCheckName}' all completed successfully.","HealthCheckName":"Debug Compilation Mode","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Result for {HealthCheckName}: {HealthCheckResult}, Message: '{HealthCheckMessage}'","HealthCheckName":"Debug Compilation Mode","HealthCheckResult":"Success","HealthCheckMessage":"Debug compilation mode is disabled.","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Checks for '{HealthCheckName}' all completed successfully.","HealthCheckName":"Cross-site scripting Protection (X-XSS-Protection header)","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Result for {HealthCheckName}: {HealthCheckResult}, Message: '{HealthCheckMessage}'","HealthCheckName":"Cross-site scripting Protection (X-XSS-Protection header)","HealthCheckResult":"Success","HealthCheckMessage":"The header X-XSS-Protection was found.","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Checks for '{HealthCheckName}' all completed successfully.","HealthCheckName":"Cookie hijacking and protocol downgrade attacks Protection (Strict-Transport-Security Header (HSTS))","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Result for {HealthCheckName}: {HealthCheckResult}, Message: '{HealthCheckMessage}'","HealthCheckName":"Cookie hijacking and protocol downgrade attacks Protection (Strict-Transport-Security Header (HSTS))","HealthCheckResult":"Success","HealthCheckMessage":"The header Strict-Transport-Security, also known as the HSTS-header, was found.","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Checks for '{HealthCheckName}' all completed successfully.","HealthCheckName":"Content/MIME Sniffing Protection","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Result for {HealthCheckName}: {HealthCheckResult}, Message: '{HealthCheckMessage}'","HealthCheckName":"Content/MIME Sniffing Protection","HealthCheckResult":"Success","HealthCheckMessage":"The header or meta-tag X-Content-Type-Options used to protect against MIME sniffing vulnerabilities was found.","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Checks for '{HealthCheckName}' all completed successfully.","HealthCheckName":"Excessive Headers","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Result for {HealthCheckName}: {HealthCheckResult}, Message: '{HealthCheckMessage}'","HealthCheckName":"Excessive Headers","HealthCheckResult":"Success","HealthCheckMessage":"No headers revealing information about the website technology were found.","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Checks for '{HealthCheckName}' all completed successfully.","HealthCheckName":"Click-Jacking Protection","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Result for {HealthCheckName}: {HealthCheckResult}, Message: '{HealthCheckMessage}'","HealthCheckName":"Click-Jacking Protection","HealthCheckResult":"Success","HealthCheckMessage":"The header or meta-tag X-Frame-Options used to control whether a site can be IFRAMEd by another was found.","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Checks for '{HealthCheckName}' all completed successfully.","HealthCheckName":"HTTPS Configuration","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Result for {HealthCheckName}: {HealthCheckResult}, Message: '{HealthCheckMessage}'","HealthCheckName":"HTTPS Configuration","HealthCheckResult":"Success","HealthCheckMessage":"You are currently viewing the site using the HTTPS scheme.","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Result for {HealthCheckName}: {HealthCheckResult}, Message: '{HealthCheckMessage}'","HealthCheckName":"HTTPS Configuration","HealthCheckResult":"Success","HealthCheckMessage":"The appSetting 'Umbraco.Core.UseHttps' is set to 'True' in your web.config file, your cookies are marked as secure.","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:23:05.0160693Z","@mt":"Result for {HealthCheckName}: {HealthCheckResult}, Message: '{HealthCheckMessage}'","HealthCheckName":"HTTPS Configuration","HealthCheckResult":"Success","HealthCheckMessage":"Your website's certificate is valid.","SourceContext":"Umbraco.Web.HealthCheck.HealthCheckResults","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T03:24:25.8185040Z","@mt":"Exception","@l":"Error","@x":"ImageProcessor.Common.Exceptions.ImageProcessingException: GetResponseAsync 132 : Response status code does not indicate success: 404 (The specified blob does not exist.).","SourceContext":"ImageProcessor.Web.Helpers.RemoteFile","ProcessId":9840,"ProcessName":"w3wp","ThreadId":29,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"ERROR"} {"@t":"2020-09-04T03:24:25.8185040Z","@mt":"Exception","@l":"Error","@x":"ImageProcessor.Common.Exceptions.ImageProcessingException: ProcessImageAsync 609 : No image exists at https://RealDomaincouk.blob.core.windows.net/media/1232/42-deep-ditches.png","SourceContext":"ImageProcessor.Web.HttpModules.ImageProcessingModule","ProcessId":9840,"ProcessName":"w3wp","ThreadId":29,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"ERROR"} {"@t":"2020-09-04T03:30:16.5455707Z","@mt":"{StartMessage} [Timing {TimingId}]","StartMessage":"Executing Umbraco RouteDefinition controller","TimingId":"110fbe1","SourceContext":"Umbraco.Web.Mvc.UmbracoPageResult","ProcessId":9840,"ProcessName":"w3wp","ThreadId":14,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"bn4uzt4yla4yiyg4mqgssbfg","HttpRequestNumber":7,"HttpRequestId":"aa352b8e-55e9-4b2c-b9c6-78ff624e8461"} {"@t":"2020-09-04T03:30:16.7007630Z","@mt":"{EndMessage} ({Duration}ms) [Timing {TimingId}]","EndMessage":"Finished","Duration":167,"TimingId":"110fbe1","SourceContext":"Umbraco.Web.Mvc.UmbracoPageResult","ProcessId":9840,"ProcessName":"w3wp","ThreadId":14,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"bn4uzt4yla4yiyg4mqgssbfg","HttpRequestNumber":7,"HttpRequestId":"aa352b8e-55e9-4b2c-b9c6-78ff624e8461"} {"@t":"2020-09-04T04:38:36.0585923Z","@mt":"{StartMessage} [Timing {TimingId}]","StartMessage":"Executing Umbraco RouteDefinition controller","TimingId":"6aa5abd","SourceContext":"Umbraco.Web.Mvc.UmbracoPageResult","ProcessId":9840,"ProcessName":"w3wp","ThreadId":18,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"f1jzlx2m0hexluk3omwlnpyt","HttpRequestNumber":8,"HttpRequestId":"a191263c-5c56-49d2-88a7-897cca792750"} {"@t":"2020-09-04T04:38:36.1991870Z","@mt":"{EndMessage} ({Duration}ms) [Timing {TimingId}]","EndMessage":"Finished","Duration":150,"TimingId":"6aa5abd","SourceContext":"Umbraco.Web.Mvc.UmbracoPageResult","ProcessId":9840,"ProcessName":"w3wp","ThreadId":18,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"f1jzlx2m0hexluk3omwlnpyt","HttpRequestNumber":8,"HttpRequestId":"a191263c-5c56-49d2-88a7-897cca792750"} {"@t":"2020-09-04T04:43:05.6352337Z","@mt":"Form Entry Submitted - Form Entry Id: {RecordId}, Form Id: {FormId}, User: {Username}","RecordId":"702ac7c6-97f1-4500-9419-010f854290b7","FormId":"d8f22a0c-3f7b-4f28-9d15-001cf12ded00","Username":"","SourceContext":"Umbraco.Forms.Core.Services.RecordService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":39,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"e5tbs322b3uh5lxxqkf1zovj","HttpRequestNumber":9,"HttpRequestId":"574a1e37-1204-441d-9854-828585557309"} {"@t":"2020-09-04T04:43:07.4242811Z","@mt":"Form Entry Approved - Form Entry Id: {RecordId}, Form Id: {FormId}, User: {Username}","RecordId":"702ac7c6-97f1-4500-9419-010f854290b7","FormId":"d8f22a0c-3f7b-4f28-9d15-001cf12ded00","Username":"","SourceContext":"Umbraco.Forms.Core.Services.RecordService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":39,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"e5tbs322b3uh5lxxqkf1zovj","HttpRequestNumber":9,"HttpRequestId":"574a1e37-1204-441d-9854-828585557309"} {"@t":"2020-09-04T05:20:46.8170286Z","@mt":"Exception","@l":"Error","@x":"ImageProcessor.Common.Exceptions.ImageProcessingException: GetResponseAsync 132 : Response status code does not indicate success: 404 (The specified blob does not exist.).","SourceContext":"ImageProcessor.Web.Helpers.RemoteFile","ProcessId":9840,"ProcessName":"w3wp","ThreadId":27,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"ERROR"} {"@t":"2020-09-04T05:20:46.8170286Z","@mt":"Exception","@l":"Error","@x":"ImageProcessor.Common.Exceptions.ImageProcessingException: ProcessImageAsync 609 : No image exists at https://RealDomaincouk.blob.core.windows.net/media/1963/RealDomain-proving-ground-4wd-transmission-and-powertrain-testing-emissions-integration-simulation.jpg","SourceContext":"ImageProcessor.Web.HttpModules.ImageProcessingModule","ProcessId":9840,"ProcessName":"w3wp","ThreadId":27,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"ERROR"} {"@t":"2020-09-04T05:21:36.1035598Z","@mt":"Error executing child request for handler 'System.Web.Mvc.HttpHandlerUtil+ServerExecuteHttpHandlerAsyncWrapper'.","SourceContext":"ASP._Page_Views_Partials_flexiblecontent__relatedCaseStudies_cshtml","ProcessId":9840,"ProcessName":"w3wp","ThreadId":23,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"oftaggh1uefbykbaagai5uyn","HttpRequestNumber":10,"HttpRequestId":"4a637854-a909-4034-94ce-522dc221d66e"} {"@t":"2020-09-04T05:45:34.3689894Z","@mt":"Exception","@l":"Error","@x":"ImageProcessor.Common.Exceptions.ImageProcessingException: GetResponseAsync 132 : Response status code does not indicate success: 404 (The specified blob does not exist.).","SourceContext":"ImageProcessor.Web.Helpers.RemoteFile","ProcessId":9840,"ProcessName":"w3wp","ThreadId":31,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"ERROR"} {"@t":"2020-09-04T05:45:34.3689894Z","@mt":"Exception","@l":"Error","@x":"ImageProcessor.Common.Exceptions.ImageProcessingException: ProcessImageAsync 609 : No image exists at https://RealDomaincouk.blob.core.windows.net/media/1204/10-mile-straight-apron.png","SourceContext":"ImageProcessor.Web.HttpModules.ImageProcessingModule","ProcessId":9840,"ProcessName":"w3wp","ThreadId":31,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"ERROR"} {"@t":"2020-09-04T06:00:53.4859949Z","@mt":"Exception","@l":"Error","@x":"ImageProcessor.Common.Exceptions.ImageProcessingException: GetResponseAsync 132 : Response status code does not indicate success: 404 (The specified blob does not exist.).","SourceContext":"ImageProcessor.Web.Helpers.RemoteFile","ProcessId":9840,"ProcessName":"w3wp","ThreadId":56,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"ERROR"} {"@t":"2020-09-04T06:00:53.4859949Z","@mt":"Exception","@l":"Error","@x":"ImageProcessor.Common.Exceptions.ImageProcessingException: ProcessImageAsync 609 : No image exists at https://RealDomaincouk.blob.core.windows.net/media/2079/RealDomain-proving-ground-motorcycle-motorbike-testing-hub-dyno.jpg","SourceContext":"ImageProcessor.Web.HttpModules.ImageProcessingModule","ProcessId":9840,"ProcessName":"w3wp","ThreadId":56,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"ERROR"} {"@t":"2020-09-04T06:13:25.4763172Z","@mt":"Error executing child request for handler 'System.Web.Mvc.HttpHandlerUtil+ServerExecuteHttpHandlerAsyncWrapper'.","SourceContext":"ASP._Page_Views_Partials_flexiblecontent__latestNews_cshtml","ProcessId":9840,"ProcessName":"w3wp","ThreadId":56,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"ox05d1fsedwdmaomswiuwntu","HttpRequestNumber":11,"HttpRequestId":"a2e08010-5819-49b3-acef-cc2bab1b8f93"} {"@t":"2020-09-04T06:20:32.5662677Z","@mt":"{StartMessage} [Timing {TimingId}]","StartMessage":"Executing Umbraco RouteDefinition controller","TimingId":"c477232","SourceContext":"Umbraco.Web.Mvc.UmbracoPageResult","ProcessId":9840,"ProcessName":"w3wp","ThreadId":37,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"vhmhewxqsgbgvpd0lyga22vp","HttpRequestNumber":12,"HttpRequestId":"d9b4627f-fede-40be-844b-6223d816df0f"} {"@t":"2020-09-04T06:20:32.7194631Z","@mt":"{EndMessage} ({Duration}ms) [Timing {TimingId}]","EndMessage":"Finished","Duration":161,"TimingId":"c477232","SourceContext":"Umbraco.Web.Mvc.UmbracoPageResult","ProcessId":9840,"ProcessName":"w3wp","ThreadId":37,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"vhmhewxqsgbgvpd0lyga22vp","HttpRequestNumber":12,"HttpRequestId":"d9b4627f-fede-40be-844b-6223d816df0f"} {"@t":"2020-09-04T06:29:26.0682257Z","@mt":"Object reference not set to an instance of an object.","SourceContext":"RealDomain.Core.Resources.ResourcesSearchService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":33,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"cu0wuq4z02pt1hvojuycgudh","HttpRequestNumber":13,"HttpRequestId":"72473791-1d18-4a9e-8de1-a24a9d4a1793"} {"@t":"2020-09-04T06:29:26.0869918Z","@mt":"Object reference not set to an instance of an object.","SourceContext":"RealDomain.Core.Resources.ResourcesSearchService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":33,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"cu0wuq4z02pt1hvojuycgudh","HttpRequestNumber":14,"HttpRequestId":"1ce66ace-0ff1-43fc-a8d9-387170d3fa56"} {"@t":"2020-09-04T06:29:41.0711863Z","@mt":"Object reference not set to an instance of an object.","SourceContext":"RealDomain.Core.Resources.ResourcesSearchService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":55,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"cu0wuq4z02pt1hvojuycgudh","HttpRequestNumber":15,"HttpRequestId":"7388df58-c698-4b8e-bb72-bbe1082d0d03"} {"@t":"2020-09-04T06:29:41.0868100Z","@mt":"Object reference not set to an instance of an object.","SourceContext":"RealDomain.Core.Resources.ResourcesSearchService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":55,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"cu0wuq4z02pt1hvojuycgudh","HttpRequestNumber":16,"HttpRequestId":"316b05dd-dcca-447a-80fa-556dd7ac911c"} {"@t":"2020-09-04T06:29:47.3645375Z","@mt":"Object reference not set to an instance of an object.","SourceContext":"RealDomain.Core.Resources.ResourcesSearchService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":59,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"cu0wuq4z02pt1hvojuycgudh","HttpRequestNumber":17,"HttpRequestId":"715d17d6-8879-438e-b7f2-46ae09f2802e"} {"@t":"2020-09-04T06:29:47.3804037Z","@mt":"Object reference not set to an instance of an object.","SourceContext":"RealDomain.Core.Resources.ResourcesSearchService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":59,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"cu0wuq4z02pt1hvojuycgudh","HttpRequestNumber":18,"HttpRequestId":"13294850-0bf9-4464-b4d9-215dcca356c5"} {"@t":"2020-09-04T06:30:04.8184121Z","@mt":"Object reference not set to an instance of an object.","SourceContext":"RealDomain.Core.Resources.ResourcesSearchService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":40,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"cu0wuq4z02pt1hvojuycgudh","HttpRequestNumber":19,"HttpRequestId":"cb2a9550-58b3-49e7-b55e-e0f97e34f6a7"} {"@t":"2020-09-04T06:30:04.8184121Z","@mt":"Object reference not set to an instance of an object.","SourceContext":"RealDomain.Core.Resources.ResourcesSearchService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":40,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"cu0wuq4z02pt1hvojuycgudh","HttpRequestNumber":20,"HttpRequestId":"c96ea771-187d-4e2a-ab94-8e169cea0cb4"} {"@t":"2020-09-04T06:30:25.2428557Z","@mt":"Object reference not set to an instance of an object.","SourceContext":"RealDomain.Core.Resources.ResourcesSearchService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":37,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"cu0wuq4z02pt1hvojuycgudh","HttpRequestNumber":21,"HttpRequestId":"02e789ca-a08e-4d56-80a4-c916f244af44"} {"@t":"2020-09-04T06:30:49.4592361Z","@mt":"Object reference not set to an instance of an object.","SourceContext":"RealDomain.Core.Resources.ResourcesSearchService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":56,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"cu0wuq4z02pt1hvojuycgudh","HttpRequestNumber":22,"HttpRequestId":"3786dc35-93e8-44cc-89e6-4c0efd862584"} {"@t":"2020-09-04T06:30:49.4592361Z","@mt":"Object reference not set to an instance of an object.","SourceContext":"RealDomain.Core.Resources.ResourcesSearchService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":56,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"cu0wuq4z02pt1hvojuycgudh","HttpRequestNumber":23,"HttpRequestId":"47352c9a-0af5-4cda-85d9-ee5dd1b1d314"} {"@t":"2020-09-04T06:30:57.2459512Z","@mt":"Object reference not set to an instance of an object.","SourceContext":"RealDomain.Core.Resources.ResourcesSearchService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":59,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"cu0wuq4z02pt1hvojuycgudh","HttpRequestNumber":24,"HttpRequestId":"e9fadcf4-497e-4eca-aad1-3843016a19f3"} {"@t":"2020-09-04T06:30:57.2615866Z","@mt":"Object reference not set to an instance of an object.","SourceContext":"RealDomain.Core.Resources.ResourcesSearchService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":59,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"cu0wuq4z02pt1hvojuycgudh","HttpRequestNumber":25,"HttpRequestId":"526eb529-9cab-4f0f-a730-17462f9136d8"} {"@t":"2020-09-04T06:31:03.0024908Z","@mt":"Object reference not set to an instance of an object.","SourceContext":"RealDomain.Core.Resources.ResourcesSearchService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":33,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"cu0wuq4z02pt1hvojuycgudh","HttpRequestNumber":26,"HttpRequestId":"041f3161-7bf0-4ed4-9506-20be8a80d1e8"} {"@t":"2020-09-04T06:31:15.4905775Z","@mt":"Object reference not set to an instance of an object.","SourceContext":"RealDomain.Core.Resources.ResourcesSearchService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":55,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"cu0wuq4z02pt1hvojuycgudh","HttpRequestNumber":27,"HttpRequestId":"957c19f5-a95d-4c55-942f-41efe71d44ec"} {"@t":"2020-09-04T06:31:15.5065038Z","@mt":"Object reference not set to an instance of an object.","SourceContext":"RealDomain.Core.Resources.ResourcesSearchService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":55,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"cu0wuq4z02pt1hvojuycgudh","HttpRequestNumber":28,"HttpRequestId":"719d2e1f-3d37-4ed6-bc18-61103e468ba7"} {"@t":"2020-09-04T06:42:14.9386212Z","@mt":"{LogPrefix} Terminating {Immediate}","LogPrefix":"[LogScrubber] ","Immediate":"","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":55,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:14.9386212Z","@mt":"{LogPrefix} Terminating {Immediate}","LogPrefix":"[KeepAlive] ","Immediate":"","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":55,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:14.9386212Z","@mt":"{LogPrefix} Terminating {Immediate}","LogPrefix":"[ServerRegistration] ","Immediate":"","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":55,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:14.9386212Z","@mt":"{LogPrefix} Terminating {Immediate}","LogPrefix":"[HealthCheckNotifier] ","Immediate":"","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":55,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:14.9386212Z","@mt":"{LogPrefix} Terminating {Immediate}","LogPrefix":"[ServerInstProcess] ","Immediate":"","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":55,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:14.9697596Z","@mt":"Stopping ({SignalSource})","SignalSource":"environment","SourceContext":"Umbraco.Core.Runtime.MainDom","ProcessId":9840,"ProcessName":"w3wp","ThreadId":55,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:15.2041353Z","@mt":"Released from MainDom","SourceContext":"Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":55,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:15.2041353Z","@mt":"{StartMessage} [Timing {TimingId}]","StartMessage":"Examine shutting down","TimingId":"eccb1bc","SourceContext":"Umbraco.Web.Search.ExamineComponent","ProcessId":9840,"ProcessName":"w3wp","ThreadId":55,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:16.6885137Z","@mt":"{EndMessage} ({Duration}ms) [Timing {TimingId}]","EndMessage":"Completed.","Duration":1485,"TimingId":"eccb1bc","SourceContext":"Umbraco.Web.Search.ExamineComponent","ProcessId":9840,"ProcessName":"w3wp","ThreadId":55,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:16.7666376Z","@mt":"Released ({SignalSource})","SignalSource":"environment","SourceContext":"Umbraco.Core.Runtime.MainDom","ProcessId":9840,"ProcessName":"w3wp","ThreadId":55,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:16.7666376Z","@mt":"{LogPrefix} Terminating {Immediate}","LogPrefix":"[ScheduledTasks] ","Immediate":"","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":55,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:16.7666376Z","@mt":"{LogPrefix} Terminating {Immediate}","LogPrefix":"[ScheduledPublishing] ","Immediate":"","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":55,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:16.7666376Z","@mt":"{LogPrefix} Terminating {Immediate}","LogPrefix":"[RebuildIndexesOnStartup] ","Immediate":"","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":55,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:16.7666376Z","@mt":"{LogPrefix} Terminating {Immediate}","LogPrefix":"[TempFileCleanup] ","Immediate":"","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":55,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:17.0010067Z","@mt":"{LogPrefix} Canceling tasks","LogPrefix":"[LogScrubber] ","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":42,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:17.0166301Z","@mt":"{LogPrefix} Tasks {TaskStatus}, terminated","LogPrefix":"[LogScrubber] ","TaskStatus":"cancelled","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":42,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:17.0166301Z","@mt":"{LogPrefix} Canceling tasks","LogPrefix":"[LogScrubber] ","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":42,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:17.4228735Z","@mt":"{LogPrefix} Canceling tasks","LogPrefix":"[KeepAlive] ","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":73,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:17.4228735Z","@mt":"{LogPrefix} Tasks {TaskStatus}, terminated","LogPrefix":"[KeepAlive] ","TaskStatus":"cancelled","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":73,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:17.9697455Z","@mt":"{LogPrefix} Canceling tasks","LogPrefix":"[KeepAlive] ","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":73,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:19.0322262Z","@mt":"{LogPrefix} Canceling tasks","LogPrefix":"[ServerRegistration] ","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":42,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:19.0322262Z","@mt":"{LogPrefix} Tasks {TaskStatus}, terminated","LogPrefix":"[ServerRegistration] ","TaskStatus":"cancelled","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":42,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:19.0322262Z","@mt":"{LogPrefix} Canceling tasks","LogPrefix":"[ServerRegistration] ","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":42,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:19.9541043Z","@mt":"{LogPrefix} Canceling tasks","LogPrefix":"[ScheduledTasks] ","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":76,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:19.9541043Z","@mt":"{LogPrefix} Tasks {TaskStatus}, terminated","LogPrefix":"[ScheduledTasks] ","TaskStatus":"cancelled","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":76,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:19.9853520Z","@mt":"{LogPrefix} Canceling tasks","LogPrefix":"[HealthCheckNotifier] ","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":73,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:19.9853520Z","@mt":"{LogPrefix} Tasks {TaskStatus}, terminated","LogPrefix":"[HealthCheckNotifier] ","TaskStatus":"cancelled","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":73,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:19.9853520Z","@mt":"{LogPrefix} Canceling tasks","LogPrefix":"[HealthCheckNotifier] ","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":73,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:19.9853520Z","@mt":"{LogPrefix} Canceling tasks","LogPrefix":"[ScheduledPublishing] ","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":73,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:19.9853520Z","@mt":"{LogPrefix} Tasks {TaskStatus}, terminated","LogPrefix":"[ScheduledPublishing] ","TaskStatus":"cancelled","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":73,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:21.0333081Z","@mt":"{LogPrefix} Canceling tasks","LogPrefix":"[ServerInstProcess] ","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":42,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:21.0333081Z","@mt":"{LogPrefix} Tasks {TaskStatus}, terminated","LogPrefix":"[ServerInstProcess] ","TaskStatus":"cancelled","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":42,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:21.0333081Z","@mt":"{LogPrefix} Canceling tasks","LogPrefix":"[ServerInstProcess] ","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":42,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:21.0333081Z","@mt":"{LogPrefix} Canceling tasks","LogPrefix":"[RebuildIndexesOnStartup] ","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":42,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:21.0333081Z","@mt":"{LogPrefix} Tasks {TaskStatus}, terminated","LogPrefix":"[RebuildIndexesOnStartup] ","TaskStatus":"cancelled","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":42,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:21.9711977Z","@mt":"{LogPrefix} Canceling tasks","LogPrefix":"[ScheduledTasks] ","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":76,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:21.9711977Z","@mt":"{LogPrefix} Canceling tasks","LogPrefix":"[TempFileCleanup] ","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":76,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:21.9711977Z","@mt":"{LogPrefix} Tasks {TaskStatus}, terminated","LogPrefix":"[TempFileCleanup] ","TaskStatus":"cancelled","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":76,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:22.0023789Z","@mt":"{LogPrefix} Canceling tasks","LogPrefix":"[ScheduledPublishing] ","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":73,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:23.0394531Z","@mt":"{LogPrefix} Canceling tasks","LogPrefix":"[RebuildIndexesOnStartup] ","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":42,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} {"@t":"2020-09-04T06:42:23.9694303Z","@mt":"{LogPrefix} Canceling tasks","LogPrefix":"[TempFileCleanup] ","SourceContext":"Umbraco.Web.Scheduling.BackgroundTaskRunner","ProcessId":9840,"ProcessName":"w3wp","ThreadId":77,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO "} ```

This second set of logs are prior to the machine switch in Azure, but I can see start ups and shut downs in it, along with application url changes. And I can see indicators of the examine indexes being out of action ({"@t":"2020-09-04T06:29:26.0682257Z","@mt":"Object reference not set to an instance of an object.","SourceContext":"RealDomain.Core.Resources.ResourcesSearchService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":33,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"cu0wuq4z02pt1hvojuycgudh","HttpRequestNumber":13,"HttpRequestId":"72473791-1d18-4a9e-8de1-a24a9d4a1793"} this error on this site indicates this (although it's unclear in the message).)

Hope this extra information is of some help and doesn't cloud the issue. I'm still waiting for the issue to re-occur on the site where I've upped the logging to debug levels.

Shazwazza commented 4 years ago

Thanks for the info. When you say examine failures in these cases, the errors are not coming from examine they are coming from custom code:

For example: 

{"@t":"2020-09-04T06:43:15.8065210Z","@mt":"Error indexing queue items","@l":"Error","@x":"System.NullReferenceException: Object reference not set to an instance of an object.\r\n   at RealDomain.Core.ExamineStartup.RegisterExamineEventsComponent.AddIndexesForCategoryTags[T](IndexingItemEventArgs e, Int32 contentId) in D:\\a\\1\\s\\src\\RealDomain.Core\\ExamineStartup\\RegisterExamineEventsComponent.cs:line 67\r\n   at RealDomain.Core.ExamineStartup.RegisterExamineEventsComponent.RegisterExamineEventsComponent_TransformingIndexValues(Object sender, IndexingItemEventArgs e) in D:\\a\\1\\s\\src\\RealDomain.Core\\ExamineStartup\\RegisterExamineEventsComponent.cs:line 106\r\n   at Umbraco.Examine.UmbracoExamineIndex.OnTransformingIndexValues(IndexingItemEventArgs e) in D:\\a\\1\\s\\src\\Umbraco.Examine\\UmbracoExamineIndex.cs:line 173\r\n   at Examine.LuceneEngine.Providers.LuceneIndex.ProcessIndexQueueItem(IndexOperation op, IndexWriter writer) in C:\\projects\\examine-qvx04\\src\\Examine\\LuceneEngine\\Providers\\LuceneIndex.cs:line 1251\r\n   at Examine.LuceneEngine.Providers.LuceneIndex.ProcessQueueItem(IndexOperation item, IndexWriter writer) in C:\\projects\\examine-qvx04\\src\\Examine\\LuceneEngine\\Providers\\LuceneIndex.cs:line 1067\r\n   at Examine.LuceneEngine.Providers.LuceneIndex.ForceProcessQueueItems(Boolean block) in C:\\projects\\examine-qvx04\\src\\Examine\\LuceneEngine\\Providers\\LuceneIndex.cs:line 913","SourceContext":"Umbraco.Examine.UmbracoContentIndex","ProcessId":4576,"ProcessName":"w3wp","ThreadId":20,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"ERROR"}

is coming from RegisterExamineEventsComponent.cs:line 106 during a TransformingIndexValues event handler.

There's also other null reference exceptions in the 2nd log from the same custom code like you say. 

{"@t":"2020-09-04T06:31:15.5065038Z","@mt":"Object reference not set to an instance of an object.","SourceContext":"RealDomain.Core.Resources.ResourcesSearchService","ProcessId":9840,"ProcessName":"w3wp","ThreadId":55,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C76798","Log4NetLevel":"INFO ","HttpSessionId":"cu0wuq4z02pt1hvojuycgudh","HttpRequestNumber":28,"HttpRequestId":"719d2e1f-3d37-4ed6-bc18-61103e468ba7"}

I don't know what the code in ResourcesSearchService is doing but it's calling something on a null object so there should be additional null checks in this custom code. Possibly during shutdown whatever resources that this service is trying to use while indexing is trying to occur during shutdown doesn't exist, or something isn't resolving a real object anymore because the app is being shutdown.

As for the URL changes, some of these should be Debug logs I think. The first one you see is just logging the fact that a request came in on a different domain since you have multiple domains, that log is here: 

{"@t":"2020-09-04T06:43:36.4156143Z","@mt":"ApplicationUrl: {UmbracoAppUrl} (using web.routing/@umbracoApplicationUrl)","UmbracoAppUrl":"https://www.RealDomain.co.uk/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":4576,"ProcessName":"w3wp","ThreadId":12,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":4,"HttpRequestId":"86886e20-a95e-44f3-b048-f0cf163cff3b"}
{"@t":"2020-09-04T06:43:43.8685992Z","@mt":"New url {Url} detected, re-discovering application url.","Url":"https://www.RealDomain.us:443/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":4576,"ProcessName":"w3wp","ThreadId":17,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":5,"HttpRequestId":"c4d4ac97-767d-48e6-b9cd-7bd534ac84ca"}

for this code: https://github.com/umbraco/Umbraco-CMS/blob/8f7615bc61c19bd80ba6e99254a69e987cbaceae/src/Umbraco.Core/RuntimeState.cs#L112

and the following log is:

{"@t":"2020-09-04T06:43:43.8685992Z","@mt":"ApplicationUrl: {UmbracoAppUrl} (using web.routing/@umbracoApplicationUrl)","UmbracoAppUrl":"https://www.RealDomain.co.uk/umbraco","SourceContext":"Umbraco.Core.Sync.ApplicationUrlHelper","ProcessId":4576,"ProcessName":"w3wp","ThreadId":17,"AppDomainId":2,"AppDomainAppId":"LMW3SVC1572358820ROOT","MachineName":"RD281878C9A228","Log4NetLevel":"INFO ","HttpRequestNumber":5,"HttpRequestId":"c4d4ac97-767d-48e6-b9cd-7bd534ac84ca"}

Saying that it's using the application url defined in web.routing/@umbracoApplicationUrl which happens here https://github.com/umbraco/Umbraco-CMS/blob/v8/contrib/src/Umbraco.Core/Sync/ApplicationUrlHelper.cs#L65

There's nothing these logs that is out of ordinary. The site looks to be shutting down correctly, MainDom is released and acquired correctly.

Shazwazza commented 4 years ago

Also note that the issue I already mentioned https://github.com/umbraco/Umbraco-CMS/pull/8772 can impact things like Transforming index values! It would be helpful for anyone to try that build mentioned above https://github.com/umbraco/Umbraco-CMS/issues/8612#issuecomment-686942505

NikRimington commented 4 years ago

No problem, I'll give the build mentioned a go this week (will try and get it applied today). When I say examine failures I mean that the indexes are not available. The Resource search service is missing a null check on the index which why there is null reference exception as this is what gets thrown when examine stops being available for me. I'll try and add some extra code in around there to check it better and put in more details into the log as well.

Shazwazza commented 4 years ago

Can you provide a code example of what is null during shutdown?

NikRimington commented 4 years ago

Normally the null reference get's thrown in the constructor which does the following. I need to add a try catch around it to better log details to try and find out what exactly is null as I didn't write this one.

public ResourcesSearchService(UmbracoHelper umbracoHelper, ILogger logger)
        {
            _umbracoHelper = umbracoHelper;
            _logger = logger ?? throw new ArgumentNullException(nameof(logger));
            if (ExamineManager.Instance.TryGetIndex(Constants.ExternalIndex, out _index))
            {
                _searcher = _index.GetSearcher();
            }
        }

I just wish I could replicate it locally to give you more information :-)

Shazwazza commented 4 years ago

UmbracoHelper is a request based lifetime, it depends on UmbracoContext which depends on HttpContext. You must not inject UmbracoHelper into any singleton, this will pin a request based lifetime object to the static root and can result in mem leaks along with all sorts of strange things happening because the UmbracoContext will get disposed at the end of each request and then you're left with a disposed instance pinned to a singleton. Please be sure you aren't doing that.

NikRimington commented 4 years ago

This service has a Request based lifetime as well so it "should" be okay. But yes, on my list of refactoring for that site is to remove the umbraco helper from being injected as I generally try to avoid as got burned by it being in odd states on the first v8 site I built :-)

umbrabot commented 2 years ago

Hiya @johnsonp57,

Just wanted to let you know that we noticed that this issue got a bit stale and might not be relevant any more.

We will close this issue for now but we're happy to open it up again if you think it's still relevant (for example: it's a feature request that's not yet implemented, or it's a bug that's not yet been fixed).

To open it this issue up again, you can write @umbrabot still relevant in a new comment as the first line. It would be super helpful for us if on the next line you could let us know why you think it's still relevant.

For example:

@umbrabot still relevant This bug can still be reproduced in version x.y.z

This will reopen the issue in the next few hours.

Thanks, from your friendly Umbraco GitHub bot :robot: :slightly_smiling_face: