search

rollbar / Rollbar.NET

Rollbar for .NET
https://docs.rollbar.com/docs/dotnet
MIT License
66 stars 44 forks source link

System.Threading.ThreadAbortException in Rollbar.dll since update to 1.0.3 #93

Closed jonathanjto closed 6 years ago

jonathanjto commented 6 years ago

Since the update to 1.0.3 version of the package the website is very slow and forearch lines of code executed (step by step) there is a HUGE number of exceptions :

Exception thrown: 'System.Threading.ThreadAbortException' in Rollbar.dll Exception thrown: 'System.Threading.ThreadAbortException' in Unknown Module.

What the hell is happening?

rivkahstandig3636 commented 6 years ago

Hi @jonathanjto we'll take a look into this right away.

akornich commented 6 years ago

@jonathanjto, can you please provide more details on how the rollbar sdk is used: type of the hosting application (webapi, webforms, wpf, etc), OS platform, flavor/version of .NET framework, specific usage scenario (if any), stack trace for the exception, etc.?

akornich commented 6 years ago

@jonathanjto, btw, as far as i remember ThreadAbortException is thrown only in response to calling Abort() on a thread. we definitely do not use Abort() within the rollbar sdk. i assume the hosting app does. here is a relevant reference for the topic: https://msdn.microsoft.com/en-us/library/system.threading.threadabortexception(v=vs.110).aspx

jonathanjto commented 6 years ago

Hi,

I completely uninstall rollbar and remove manually the rollbar.dll and it seems to work now. I'll keep you in touch.

The website runs under IIS on Windows 10 and I debug it with Visual Studio 2017.

Here is the list of packages included in the ASP.NET MVC application.


<packages>
  <package id="Antlr" version="3.5.0.2" targetFramework="net452" />
  <package id="bootstrap" version="4.0.0" targetFramework="net452" />
  <package id="EntityFramework" version="6.2.0" targetFramework="net452" />
  <package id="jQuery" version="3.3.1" targetFramework="net452" />
  <package id="Microsoft.AspNet.Identity.Core" version="2.2.1" targetFramework="net452" />
  <package id="Microsoft.AspNet.Identity.EntityFramework" version="2.2.1" targetFramework="net452" />
  <package id="Microsoft.AspNet.Identity.Owin" version="2.2.1" targetFramework="net452" />
  <package id="Microsoft.AspNet.Mvc" version="5.2.4" targetFramework="net452" />
  <package id="Microsoft.AspNet.Mvc.fr" version="5.2.4" targetFramework="net452" />
  <package id="Microsoft.AspNet.Razor" version="3.2.4" targetFramework="net452" />
  <package id="Microsoft.AspNet.Razor.fr" version="3.2.4" targetFramework="net452" />
  <package id="Microsoft.AspNet.SignalR" version="2.2.2" targetFramework="net452" />
  <package id="Microsoft.AspNet.SignalR.Client" version="2.2.2" targetFramework="net452" />
  <package id="Microsoft.AspNet.SignalR.Core" version="2.2.2" targetFramework="net452" />
  <package id="Microsoft.AspNet.SignalR.Core.fr" version="2.2.2" targetFramework="net452" />
  <package id="Microsoft.AspNet.SignalR.fr" version="2.2.2" targetFramework="net452" />
  <package id="Microsoft.AspNet.SignalR.JS" version="2.2.2" targetFramework="net452" />
  <package id="Microsoft.AspNet.SignalR.SystemWeb" version="2.2.2" targetFramework="net452" />
  <package id="Microsoft.AspNet.SignalR.SystemWeb.fr" version="2.2.2" targetFramework="net452" />
  <package id="Microsoft.AspNet.Web.Optimization" version="1.1.3" targetFramework="net452" />
  <package id="Microsoft.AspNet.WebApi.Client" version="5.2.4" targetFramework="net452" />
  <package id="Microsoft.AspNet.WebPages" version="3.2.4" targetFramework="net452" />
  <package id="Microsoft.AspNet.WebPages.fr" version="3.2.4" targetFramework="net452" />
  <package id="Microsoft.CodeDom.Providers.DotNetCompilerPlatform" version="1.0.8" targetFramework="net452" />
  <package id="Microsoft.Net.Compilers" version="2.6.1" targetFramework="net452" developmentDependency="true" />
  <package id="Microsoft.NETCore.Platforms" version="2.0.1" targetFramework="net452" />
  <package id="Microsoft.Owin" version="4.0.0" targetFramework="net452" />
  <package id="Microsoft.Owin.fr" version="4.0.0" targetFramework="net452" />
  <package id="Microsoft.Owin.Host.SystemWeb" version="4.0.0" targetFramework="net452" />
  <package id="Microsoft.Owin.Host.SystemWeb.fr" version="4.0.0" targetFramework="net452" />
  <package id="Microsoft.Owin.Security" version="4.0.0" targetFramework="net452" />
  <package id="Microsoft.Owin.Security.Cookies" version="4.0.0" targetFramework="net452" />
  <package id="Microsoft.Owin.Security.fr" version="4.0.0" targetFramework="net452" />
  <package id="Microsoft.Owin.Security.OAuth" version="4.0.0" targetFramework="net452" />
  <package id="Microsoft.Web.Infrastructure" version="1.0.0.0" targetFramework="net452" />
  <package id="Modernizr" version="2.8.3" targetFramework="net452" />
  <package id="MvcDonutCaching" version="1.3.0" targetFramework="net452" />
  <package id="NETStandard.Library" version="2.0.1" targetFramework="net452" />
  <package id="Newtonsoft.Json" version="11.0.1" targetFramework="net452" />
  <package id="Owin" version="1.0" targetFramework="net452" />
  <package id="popper.js" version="1.12.9" targetFramework="net452" />
  <package id="RecaptchaNet" version="2.1.0" targetFramework="net452" />
  <package id="Rollbar" version="1.0.3" targetFramework="net452" />
  <package id="RouteLocalization.Mvc" version="2.3.0" targetFramework="net452" />
  <package id="System.Collections" version="4.3.0" targetFramework="net452" />
  <package id="System.Collections.Concurrent" version="4.3.0" targetFramework="net452" />
  <package id="System.Diagnostics.Debug" version="4.3.0" targetFramework="net452" />
  <package id="System.Diagnostics.Tools" version="4.3.0" targetFramework="net452" />
  <package id="System.Diagnostics.Tracing" version="4.3.0" targetFramework="net452" />
  <package id="System.Globalization" version="4.3.0" targetFramework="net452" />
  <package id="System.IO" version="4.3.0" targetFramework="net452" />
  <package id="System.IO.Compression" version="4.3.0" targetFramework="net452" />
  <package id="System.Linq" version="4.3.0" targetFramework="net452" />
  <package id="System.Linq.Expressions" version="4.3.0" targetFramework="net452" />
  <package id="System.Net.Http" version="4.3.3" targetFramework="net452" />
  <package id="System.Net.Primitives" version="4.3.0" targetFramework="net452" />
  <package id="System.ObjectModel" version="4.3.0" targetFramework="net452" />
  <package id="System.Reflection" version="4.3.0" targetFramework="net452" />
  <package id="System.Reflection.Extensions" version="4.3.0" targetFramework="net452" />
  <package id="System.Reflection.Primitives" version="4.3.0" targetFramework="net452" />
  <package id="System.Resources.ResourceManager" version="4.3.0" targetFramework="net452" />
  <package id="System.Runtime" version="4.3.0" targetFramework="net452" />
  <package id="System.Runtime.Extensions" version="4.3.0" targetFramework="net452" />
  <package id="System.Runtime.InteropServices" version="4.3.0" targetFramework="net452" />
  <package id="System.Runtime.InteropServices.RuntimeInformation" version="4.3.0" targetFramework="net452" />
  <package id="System.Runtime.Numerics" version="4.3.0" targetFramework="net452" />
  <package id="System.Text.Encoding" version="4.3.0" targetFramework="net452" />
  <package id="System.Text.Encoding.Extensions" version="4.3.0" targetFramework="net452" />
  <package id="System.Text.RegularExpressions" version="4.3.0" targetFramework="net452" />
  <package id="System.Threading" version="4.3.0" targetFramework="net452" />
  <package id="System.Threading.Tasks" version="4.3.0" targetFramework="net452" />
  <package id="System.Threading.Timer" version="4.3.0" targetFramework="net452" />
  <package id="System.Xml.ReaderWriter" version="4.3.1" targetFramework="net452" />
  <package id="System.Xml.XDocument" version="4.3.0" targetFramework="net452" />
  <package id="Telegram.Bot" version="13.4.0" targetFramework="net452" />
  <package id="WebGrease" version="1.6.0" targetFramework="net452" />
</packages>
jonathanjto commented 6 years ago

So I've tried a lots of stuff this morning...

I debug, stop debug, debug again, ... and the problem occurs. Cleaning the app has no effect.

If i remove bin/release folder and restart IIS and it's working once in debug mode.

Then I modify something and relaunch debug I've got only "Exception thrown: 'System.Threading.ThreadAbortException' in Unknown Module."

Then I change something and relaunch and now I've Exception thrown: 'System.Threading.ThreadAbortException' in Rollbar.dll Exception thrown: 'System.Threading.ThreadAbortException' in Unknown Module.

For now i've removed all lines of code involving Rollbar, so the only line of code used for rollbar is : RollbarLocator.RollbarInstance.Configure(new RollbarConfig(postServerItemAccessToken) { Environment = "DEV", Enabled = false });

If I remove this line the problem does not occurs anymore.

It's getting weirder and weirder ... In another MVC app in the same Visual Studio Solution using almost the same list of packages the problem is not occurring ... (ouf, that one is in production the other one not).

The only three packages that are added to the project with problems are: 

<package id="MvcDonutCaching" version="1.3.0" targetFramework="net452" />
<package id="RecaptchaNet" version="2.1.0" targetFramework="net452" />
<package id="RouteLocalization.Mvc" version="2.3.0" targetFramework="net452" />

Another stuff is the use of a custom HTTP Module managing routing (but should not be a problem). The routing system has been completely rewritten in the application with problem (support for localization and custom URL) - I suggest rollbar has a problem with that OR with the cache system but the cache is disable in dev...

PS: apologize my english, it's not my mother language

akornich commented 6 years ago

@jonathanjto, interesting case you got out there. May I ask you to make sure that the ThreadAbortException is checked within Visual Studio's Exception Settings and try o repro the problem again? This time Visual Studio debugger should be able to pin point the source of the exception throw.

jonathanjto commented 6 years ago

@akornich here's all information i've got

First exception : Exception thrown: 'System.Threading.ThreadAbortException' in Unknown Module. Your app has entered a break state, but no code is currently executing that is supported by the selected debug engine (e.g. only native runtime code is executing).

Rollbar exception : untitled

System.Threading.ThreadAbortException: 'Le thread a été abandonné.' C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\root\0243d796\233b9d49\assembly\dl3\5c2e41ab\00962077_1aabd301\Rollbar.dll

--- Already tried to clone the repository to get the pdb but I did not have all framework version required to build the app :'(

akornich commented 6 years ago

@jonathanjto, you can always open SolutionCommon.csproj file and replace <TargetFrameworks>netcoreapp2.0;net471;net47;net462;net461;net46;net452;net451;net45;netstandard2.0</TargetFrameworks> with <TargetFramework>net452</TargetFramework> to build it with only one .net version you need...

akornich commented 6 years ago

@jonathanjto, one more area to review is if you assign any of the custom payload filtering or transforming delegates on the RollbarConfig object. If you do, make sure they run clean.

jonathanjto commented 6 years ago

@akornich

No transformation that i'm aware of... Errors are thrown even before the "Application Start" calls.

I've tried to build the DLL from your projects in debug mode with debugging information to full and if I used it it's working ... ????

Impossible to link the PDB file to the nuget DDL.

This is hell

akornich commented 6 years ago

@jonathanjto, can you, please, clarify: are you saying when you build rollbar sdk directly from github clone - everything works well? Please, try clearing all the NuGet caches and reintegrating rollbar sdk from Nuget again. It sounds as you got a corrupt assembly in the first place.

jonathanjto commented 6 years ago

@akornich i'll try again to build again from a cloned repository but yes i'm pretty sure it was working (dll in debug mode)

I've tried to clear all nuget caches it's not working

akornich commented 6 years ago

@jonathanjto, please, try using release configuration of rollbar.net when you do it by building the github clone again. meanwhile, i'll try reconfiguring one of our samples to use nuget instead of the direct project references and see if can repro your problem.

akornich commented 6 years ago

@jonathanjto, a quick update for you: the asp.net core web api sample that we have runs fine against nuget 1.0.3 package. I also created a sample web api app based on .net 4.5.2 with rollbar.net 1.0.3 from nuget and it runs smooth. Of cause, there could be something specific about your project that results in error, but taking in account that it all works when you build rollbar 1.0.3 sdk from github clone, it all leads me to believe you are experiencing some sort of local nuget client/manager package registration issue.

trygvelo commented 6 years ago

We have a similar project with the same issue. The issue has been going on since we installed Rollbar 1.0.1 in the project. Version 1.0.3 has the same symptoms. We believe it is something with SignalR and/or Rollbar or a combination of the two not working well.

Every time we do a redeployment, CPU for the w3wp.exe process goes into a constant spike.

IF however we make sure ALL browser windows are closed, ie. NO open SignalR connections BEFORE deployment, it works fine without high CPU.

Doing a memory dump and analyzing it with windbg shows:

Thread 26: 26 12 25d8 000000000d975540 1002b221 Preemptive 0000000303059A70:000000030305AD68 00000000028f5060 0 MTA System.Threading.ThreadAbortException 0000000303059888

0:026> !clrstack OS Thread Id: 0x25d8 (26) Child SP IP Call Site 000000000eaceba8 000007fef960e226 [HelperMethodFrame_1OBJ: 000000000eaceba8] System.Threading.Monitor.Enter(System.Object) 000000000eaceca0 000007fe9a41e722 Rollbar.RollbarQueueController.KeepProcessingAllQueues() 000000000eacecf0 000007fef83f4740 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean) [f:\dd\ndp\clr\src\BCL\system\threading\executioncontext.cs @ 954] 000000000eacedc0 000007fef83f45d4 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean) [f:\dd\ndp\clr\src\BCL\system\threading\executioncontext.cs @ 902] 000000000eacedf0 000007fef83f45a2 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [f:\dd\ndp\clr\src\BCL\system\threading\executioncontext.cs @ 891] 000000000eacee40 000007fef847cf62 System.Threading.ThreadHelper.ThreadStart() [f:\dd\ndp\clr\src\BCL\system\threading\thread.cs @ 111] 000000000eacf098 000007fef9605a03 [GCFrame: 000000000eacf098] 000000000eacf3e8 000007fef9605a03 [DebuggerU2MCatchHandlerFrame: 000000000eacf3e8] 000000000eacf578 000007fef9605a03 [ContextTransitionFrame: 000000000eacf578] 000000000eacf7a8 000007fef9605a03 [DebuggerU2MCatchHandlerFrame: 000000000eacf7a8]

THEN !PrintException:

0:026> !PrintException /d 0000000303059888 Exception object: 0000000303059888 Exception type: System.Threading.ThreadAbortException Message: Thread was being aborted. InnerException: StackTrace (generated): SP IP Function 000000000EACECA0 0000000000000000 mscorlib_ni!System.Threading.Monitor.Enter(System.Object)+0x1 000000000EACECA0 000007FE9A41E722 UNKNOWN!Rollbar.RollbarQueueController.KeepProcessingAllQueues()+0x42

StackTraceString: HResult: 80131530

jonathanjto commented 6 years ago

@trygvelo nice to see that i'm not the only one... For now I fix this using the DLL built in debug mode from the repository because i've no time to investigate further...

trygvelo commented 6 years ago

When investigating further it is obvious that there are TWO threads doing Rollbar.RollbarQueueController.KeepProcessingAllQueues(). Both casting ThreadAbortExceptions:

   ID OSID ThreadOBJ           State GC Mode     GC Alloc Context                  Domain           Count Apt Exception

7 1 95c 00000000027aef20 28220 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 Ukn 20 2 40fc 0000000002863410 2b220 Preemptive 000000020105AA98:000000020105B550 00000000027a71d0 0 MTA (Finalizer) 22 3 6e84 00000000028dc2f0 102a220 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 MTA (Threadpool Worker) 23 4 47f0 00000000028f60a0 2021220 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 Ukn 24 7 37a0 000000000d6bfd90 8029220 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 MTA (Threadpool Completion Port) 25 11 49f4 000000000d9743f0 1020220 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 Ukn (Threadpool Worker) 26 12 25d8 000000000d975540 1002b221 Preemptive 0000000303059A70:000000030305AD68 00000000028f5060 0 MTA System.Threading.ThreadAbortException 0000000303059888 XXXX 15 0 000000000da101d0 1039820 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 MTA (Threadpool Worker) XXXX 16 0 000000000da11a80 1039820 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 MTA (Threadpool Worker) 28 17 3574 000000000f95e600 21220 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 Ukn 29 18 1edc 000000000f95edd0 21220 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 Ukn 30 19 2678 000000000f95f5a0 21220 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 Ukn 31 20 6e08 000000000f95fd70 21220 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 Ukn XXXX 22 0 000000000fb00110 39820 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 Ukn 2 5 5a74 000000000fb008e0 20220 Preemptive 000000020106D5B0:000000020106F550 00000000027a71d0 0 Ukn 35 266 3be4 000000000fe8a3a0 1029220 Preemptive 000000020105D7C0:000000020105F550 00000000027a71d0 0 MTA (Threadpool Worker) 36 126 3a10 000000000fbb87d0 1029220 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 MTA (Threadpool Worker) 37 133 2ef8 000000000fe8b400 1029220 Preemptive 0000000201061D18:0000000201063550 00000000027a71d0 0 MTA (Threadpool Worker) 38 136 17dc 000000000ff10fc0 3029220 Preemptive 0000000201069920:000000020106B550 000000000d6d46a0 0 MTA (Threadpool Worker) 39 138 1564 000000000fe97a50 1029220 Preemptive 0000000300F31318:0000000300F32D68 00000000027a71d0 0 MTA (Threadpool Worker) 44 143 41a4 000000000fdfe510 1029220 Preemptive 000000030195D420:000000030195ED68 00000000027a71d0 0 MTA (Threadpool Worker) 40 144 382c 000000000fe989f0 1029220 Preemptive 00000002010658A0:0000000201067550 00000000027a71d0 0 MTA (Threadpool Worker) 42 145 3a18 000000000ff11790 1029220 Preemptive 0000000201067A98:0000000201069550 00000000027a71d0 0 MTA (Threadpool Worker) 41 146 55ac 000000000fe1e1c0 1029220 Preemptive 0000000300F34FC0:0000000300F36D68 00000000027a71d0 0 MTA (Threadpool Worker) 43 147 2aa4 000000000fee5460 1029220 Preemptive 0000000102D9D318:0000000102D9E860 00000000027a71d0 0 MTA (Threadpool Worker) 45 148 544c 000000000ff11f60 1029220 Preemptive 0000000404BCAAF8:0000000404BCC960 00000000027a71d0 0 MTA (Threadpool Worker) 46 149 5228 000000000fe77340 1029220 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 MTA (Threadpool Worker) 47 150 5f18 000000000fdfece0 1029220 Preemptive 0000000300F33660:0000000300F34D68 00000000027a71d0 0 MTA (Threadpool Worker) 48 152 1740 000000000fe991c0 1029220 Preemptive 0000000300F2F980:0000000300F30D68 00000000027a71d0 0 MTA (Threadpool Worker) 49 153 6d9c 000000000fdabb60 1029220 Preemptive 000000020105F740:0000000201061550 00000000027a71d0 0 MTA (Threadpool Worker) 50 141 55c 000000000fe590a0 3029220 Preemptive 0000000201063CD8:0000000201065550 00000000028f5060 0 MTA (Threadpool Worker) 51 184 4088 000000000fe79340 1029220 Preemptive 0000000102D95780:0000000102D96860 00000000027a71d0 0 MTA (Threadpool Worker) 52 288 5f44 000000001003cba0 20220 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 Ukn 6 279 6708 000000000fe7eb10 20220 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 Ukn XXXX 293 0 000000000ffdcde0 8039820 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 MTA (Threadpool Completion Port) 54 294 1bbc 000000000fe7f2e0 1002b221 Cooperative 0000000405125438:0000000405126960 000000000d6d46a0 0 MTA System.Threading.ThreadAbortException 0000000405125250 XXXX 295 0 000000000fe7fab0 39820 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 Ukn XXXX 296 0 000000000ff8d4b0 39820 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 Ukn XXXX 297 0 000000001254c880 39820 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 MTA 55 298 1458 000000000fe79b10 1029220 Preemptive 0000000101D83468:0000000101D84860 00000000027a71d0 0 MTA (Threadpool Worker) 56 290 300c 000000000ff107f0 202b220 Preemptive 0000000201070508:0000000201071550 000000000fe233c0 0 MTA 57 289 2630 000000000fe7a2e0 202b220 Preemptive 0000000404FF0D30:0000000404FF2960 000000000fe233c0 1 MTA 53 287 51ec 000000000fdfdd40 20220 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 Ukn 58 292 6304 000000000ff8cce0 202b220 Preemptive 0000000000000000:0000000000000000 000000000fe233c0 0 MTA XXXX 286 0 000000000fe7aab0 39820 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 Ukn XXXX 285 0 000000000fe8c3a0 8039820 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 MTA (Threadpool Completion Port) XXXX 284 0 000000000ffdbe40 39820 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 Ukn XXXX 291 0 000000000ffdc610 39820 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 Ukn 59 283 595c 000000000ffdb670 8029220 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 MTA (Threadpool Completion Port) 60 282 329c 000000000fee4c90 8029220 Preemptive 0000000000000000:0000000000000000 00000000027a71d0 0 MTA (Threadpool Completion Port)

Thread number 26 and 54. This is the PrintException output of both threads:

0:026> !PrintException /d 0000000303059888 Exception object: 0000000303059888 Exception type: System.Threading.ThreadAbortException Message: Thread was being aborted. InnerException: StackTrace (generated): SP IP Function 000000000EACECA0 0000000000000000 mscorlib_ni!System.Threading.Monitor.Enter(System.Object)+0x1 000000000EACECA0 000007FE9A41E722 UNKNOWN!Rollbar.RollbarQueueController.KeepProcessingAllQueues()+0x42

StackTraceString: HResult: 80131530 0:026> !PrintException /d 0000000405125250 Exception object: 0000000405125250 Exception type: System.Threading.ThreadAbortException Message: Thread was being aborted. InnerException: StackTrace (generated): SP IP Function 00000000114EEEF0 0000000000000000 mscorlib_ni!System.Threading.Monitor.Enter(System.Object)+0x1 00000000114EEEF0 000007FE9BEFE6B2 UNKNOWN!Rollbar.RollbarQueueController.KeepProcessingAllQueues()+0x42

StackTraceString: HResult: 80131530

geoffappleford commented 6 years ago

I'm also experiencing the same problem using version 1.0.3 from nuget. Using the DLL built from the source in debug mode "fixes the problem". Building in release mode exhibits the same high cpu behaviour as the nuget DLL.

I see this in production using IIS and dev using IIS Express.

Like @trygvelo, I am also using SignalR. However I tried recreating the issue in a new basic project with SignalR but couldn't.

akornich commented 6 years ago

Thanks to everyone supplying the details! I am still struggling to repro the problem on my side. Meantime, I would like to summarize what we know/suspect so far:

Here are a few questions:

jonathanjto commented 6 years ago

Hello

geoffappleford commented 6 years ago

Ok, this is strange...

I'm on Windows, .NET 4.7 Asp.net MVC project. Using 1.0.3.

I don't think SignalR is involved.

I can reliably reproduce the problem (on my computer anyway) by following the steps below. I've tried a few blank applications and it also repros on the Rollbar Webforms sample app. (which I see is using Rollbar 1.0.1)

  1. Create a new blank MVC application
  2. Setup Rollbar (already set up in the Webforms app)
  3. Run the application
  4. Add/remove any nuget package

As soon as I remove a nuget package, the CPU usage climbs to about 30%. Remove another and it goes to 60%, another 90%...

As I said...weird...

akornich commented 6 years ago

@geoffappleford, the steps you described cause excessive CPU usage. What about the System.Threading.ThreadAbortException? Are you getting it as well? Also, I derive that on the step 3 the app runs fine and every time you do step 4 you do full rebuild/redeploy. Correct?

akornich commented 6 years ago

Gentlemen, since I am not able to repro the problem, may I ask all of you whoever has Rollbar.NET integration via source code clone to try swapping in RollbarQueueController.cs the existing singleton implementation with this one? :

    #region singleton implementation

    private static volatile RollbarQueueController instance;
    private static object syncRoot = new Object();

    private RollbarQueueController()
    {
        this._rollbarCommThread = new Thread(this.KeepProcessingAllQueues)
        {
            IsBackground = true,
            Name = "Rollbar Communication Thread"
        };

        this._rollbarCommThread.Start();
    }

    public static RollbarQueueController Instance
    {
        get
        {
            if (instance == null)
            {
                lock (syncRoot)
                {
                    if (instance == null)
                        instance = new RollbarQueueController();
                }
            }

            return instance;
        }
    }

    #endregion singleton implementation

by just replacing the whole code block of the #region singleton implementation.

And then try to repro the problem in the release build. Thank you!

geoffappleford commented 6 years ago

Swapping in the code above does not make a difference.

@akornich - I am also seeing the System.Threading.ThreadAbortException. The high CPU is because this exception is thrown repeatedly (and caught in the try block ) in KeepProcessingAllQueues in RollbarQueueController.cs. And no rebuild/redeploy is necessary after step 4.

Interestingly, if I remove the try/catch block I don't get the runaway high cpu. It seems that the exception is still thrown a few times but then the app is reloaded or restarted. At least I think that's what's happening. Here's and extract from the VS debug log (debugging webforms app after referencing rollbar project directly instead of via nuget) as soon as I add/remove a nuget package. 

Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
The thread 0x4c84 has exited with code 0 (0x0).
The thread 0x4c80 has exited with code 0 (0x0).
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
The thread 0x4c74 has exited with code 0 (0x0).
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
Exception thrown: 'System.Threading.ThreadAbortException' in System.Web.dll
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
'iisexpress.exe' (CLR v4.0.30319: Domain 2): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll'
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650564512798595): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\System.Web\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Web.dll'
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650564512798595): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll'
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650564512798595): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Core\v4.0_4.0.0.0__b77a5c561934e089\System.Core.dll'
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650564512798595): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Web.ApplicationServices\v4.0_4.0.0.0__31bf3856ad364e35\System.Web.ApplicationServices.dll'
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650564512798595): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Configuration\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Configuration.dll'
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650564512798595): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Xml\v4.0_4.0.0.0__b77a5c561934e089\System.Xml.dll'
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650564512798595): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Runtime.Caching\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Runtime.Caching.dll'
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650564512798595): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\Microsoft.Build.Utilities.v4.0\v4.0_4.0.0.0__b03f5f7f11d50a3a\Microsoft.Build.Utilities.v4.0.dll'
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650564512798595): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\Microsoft.JScript\v4.0_10.0.0.0__b03f5f7f11d50a3a\Microsoft.JScript.dll'
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650564512798595): Unloaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\c5a7c1bb\00449e52_7846d301\Microsoft.CodeDom.Providers.DotNetCompilerPlatform.dll'
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650564512798595): Unloaded

Here's the same extract with the try block removed:

Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
Exception thrown: 'System.Threading.ThreadAbortException' in Rollbar.dll
The thread 0x4e98 has exited with code 0 (0x0).
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
Exception thrown: 'System.Threading.ThreadAbortException' in Rollbar.dll
Exception thrown: 'System.Threading.ThreadAbortException' in Rollbar.dll
The thread 0x4760 has exited with code 0 (0x0).
Exception thrown: 'System.Threading.ThreadAbortException' in Rollbar.dll
Exception thrown: 'System.Threading.ThreadAbortException' in Rollbar.dll
Exception thrown: 'System.Threading.ThreadAbortException' in Rollbar.dll
Exception thrown: 'System.Threading.ThreadAbortException' in Rollbar.dll
Exception thrown: 'System.Threading.ThreadAbortException' in Rollbar.dll

That then goes on seemingly indefinitely.

Here's the full debug log.

'iisexpress.exe' (CLR v4.0.30319: DefaultDomain): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: DefaultDomain): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\System.Web\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Web.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: DefaultDomain): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: DefaultDomain): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Core\v4.0_4.0.0.0__b77a5c561934e089\System.Core.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: DefaultDomain): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Web.ApplicationServices\v4.0_4.0.0.0__31bf3856ad364e35\System.Web.ApplicationServices.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: DefaultDomain): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Configuration\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Configuration.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: DefaultDomain): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Xml\v4.0_4.0.0.0__b77a5c561934e089\System.Xml.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: Domain 2): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\System.Web\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Web.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Core\v4.0_4.0.0.0__b77a5c561934e089\System.Core.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Web.ApplicationServices\v4.0_4.0.0.0__31bf3856ad364e35\System.Web.ApplicationServices.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Configuration\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Configuration.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Xml\v4.0_4.0.0.0__b77a5c561934e089\System.Xml.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Runtime.Caching\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Runtime.Caching.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: DefaultDomain): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Runtime.Caching\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Runtime.Caching.dll'. Cannot find or open the PDB file.
Exception thrown: 'System.UnauthorizedAccessException' in mscorlib.dll
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\Microsoft.Build.Utilities.v4.0\v4.0_4.0.0.0__b03f5f7f11d50a3a\Microsoft.Build.Utilities.v4.0.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: DefaultDomain): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\Microsoft.Build.Utilities.v4.0\v4.0_4.0.0.0__b03f5f7f11d50a3a\Microsoft.Build.Utilities.v4.0.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\Microsoft.JScript\v4.0_10.0.0.0__b03f5f7f11d50a3a\Microsoft.JScript.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\c5a7c1bb\00449e52_7846d301\Microsoft.CodeDom.Providers.DotNetCompilerPlatform.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\Microsoft.VisualStudio.Web.PageInspector.Loader\v4.0_1.0.0.0__b03f5f7f11d50a3a\Microsoft.VisualStudio.Web.PageInspector.Loader.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\Microsoft.CSharp\v4.0_4.0.0.0__b03f5f7f11d50a3a\Microsoft.CSharp.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\System.Data\v4.0_4.0.0.0__b77a5c561934e089\System.Data.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Web.Services\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Web.Services.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Drawing\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Drawing.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\System.EnterpriseServices\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.EnterpriseServices.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.IdentityModel\v4.0_4.0.0.0__b77a5c561934e089\System.IdentityModel.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Runtime.Serialization\v4.0_4.0.0.0__b77a5c561934e089\System.Runtime.Serialization.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.ServiceModel\v4.0_4.0.0.0__b77a5c561934e089\System.ServiceModel.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.ServiceModel.Activation\v4.0_4.0.0.0__31bf3856ad364e35\System.ServiceModel.Activation.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.ServiceModel.Web\v4.0_4.0.0.0__31bf3856ad364e35\System.ServiceModel.Web.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Activities\v4.0_4.0.0.0__31bf3856ad364e35\System.Activities.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.ServiceModel.Activities\v4.0_4.0.0.0__31bf3856ad364e35\System.ServiceModel.Activities.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.WorkflowServices\v4.0_4.0.0.0__31bf3856ad364e35\System.WorkflowServices.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Web.Extensions\v4.0_4.0.0.0__31bf3856ad364e35\System.Web.Extensions.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Data.DataSetExtensions\v4.0_4.0.0.0__b77a5c561934e089\System.Data.DataSetExtensions.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Xml.Linq\v4.0_4.0.0.0__b77a5c561934e089\System.Xml.Linq.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.ComponentModel.DataAnnotations\v4.0_4.0.0.0__31bf3856ad364e35\System.ComponentModel.DataAnnotations.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Web.DynamicData\v4.0_4.0.0.0__31bf3856ad364e35\System.Web.DynamicData.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\d52fd917\00600d23_32aece01\Antlr3.Runtime.dll'. Symbols loaded.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\d9d1bfe4\004a7359_1c08d201\AspNet.ScriptManager.bootstrap.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\e304266e\00f89843_a318d201\AspNet.ScriptManager.jQuery.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\50c51b76\00afbf75_34efd201\Microsoft.AI.Agent.Intercept.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\bfd59044\00ef6228_37f7d201\Microsoft.AI.DependencyCollector.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\89639b66\00b200f9_37f7d201\Microsoft.AI.PerfCounterCollector.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\68940dcf\00833d7d_49efd201\Microsoft.AI.ServerTelemetryChannel.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\f38a3a2c\0085cff7_37f7d201\Microsoft.AI.Web.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\ff9920db\00b200f9_37f7d201\Microsoft.AI.WindowsServer.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\9ab34299\0082b8e9_49efd201\Microsoft.ApplicationInsights.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\28e0f7ec\0069051d_358dce01\Microsoft.AspNet.FriendlyUrls.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\515eff03\002f8135_0ff0d201\Microsoft.AspNet.TelemetryCorrelation.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\0d668c2f\003e13d0_2c27cf01\Microsoft.AspNet.Web.Optimization.WebForms.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\c76914c6\002eee49_e2aace01\Microsoft.ScriptManager.MSAjax.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\1517bc3f\000fe44f_e2aace01\Microsoft.ScriptManager.WebForms.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\f7edda1d\00b4f9b4_4a6acd01\Microsoft.Web.Infrastructure.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\f16286d1\00aff703_2ae8d201\Newtonsoft.Json.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\1008d2d4\262b66d9_7db7d301\Rollbar.dll'. Symbols loaded.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\34d3a4ee\3237dde5_7db7d301\Sample.WebForms.Site.dll'. Symbols loaded.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\c1c35a48\0029ad3a_6500d301\System.Diagnostics.DiagnosticSource.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\b9b482fb\003e13d0_2c27cf01\System.Web.Optimization.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\72fac1a2\004b384d_3218cf01\WebGrease.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Xaml\v4.0_4.0.0.0__b77a5c561934e089\System.Xaml.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\Extensions\Microsoft\Web Tools\Page Inspector\Microsoft.VisualStudio.Web.PageInspector.Runtime.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\Extensions\Microsoft\Web Tools\Page Inspector\Microsoft.VisualStudio.Web.PageInspector.Tracing.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Web.RegularExpressions\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Web.RegularExpressions.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: DefaultDomain): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Web.RegularExpressions\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Web.RegularExpressions.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\System.EnterpriseServices\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.EnterpriseServices.Wrapper.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Data.Services.Design\v4.0_4.0.0.0__b77a5c561934e089\System.Data.Services.Design.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Runtime\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Runtime.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\App_global.asax.bmlfllso.dll'. 
'iisexpress.exe' (CLR v4.0.30319: Domain 3): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: AppInsightsDomain-19f2071b-dd03-4c5a-9873-ded2cd0d4d19): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\System.Web\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Web.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: AppInsightsDomain-19f2071b-dd03-4c5a-9873-ded2cd0d4d19): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\ff9920db\00b200f9_37f7d201\Microsoft.AI.WindowsServer.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: AppInsightsDomain-19f2071b-dd03-4c5a-9873-ded2cd0d4d19): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\Microsoft.WindowsAzure.ServiceRuntime\v4.0_2.7.0.0__31bf3856ad364e35\Microsoft.WindowsAzure.ServiceRuntime.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: AppInsightsDomain-19f2071b-dd03-4c5a-9873-ded2cd0d4d19): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Xml.Linq\v4.0_4.0.0.0__b77a5c561934e089\System.Xml.Linq.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: AppInsightsDomain-19f2071b-dd03-4c5a-9873-ded2cd0d4d19): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: AppInsightsDomain-19f2071b-dd03-4c5a-9873-ded2cd0d4d19): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Core\v4.0_4.0.0.0__b77a5c561934e089\System.Core.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: AppInsightsDomain-19f2071b-dd03-4c5a-9873-ded2cd0d4d19): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Configuration\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Configuration.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: AppInsightsDomain-19f2071b-dd03-4c5a-9873-ded2cd0d4d19): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Xml\v4.0_4.0.0.0__b77a5c561934e089\System.Xml.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: AppInsightsDomain-19f2071b-dd03-4c5a-9873-ded2cd0d4d19): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\msshrtmi\v4.0_2.7.0.0__31bf3856ad364e35\msshrtmi.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: DefaultDomain): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\msshrtmi\v4.0_2.7.0.0__31bf3856ad364e35\msshrtmi.dll'. Cannot find or open the PDB file.
Microsoft.WindowsAzure.ServiceRuntime Information: 100 : Role environment . INITIALIZING
Exception thrown: 'System.Security.SecurityException' in mscorlib.dll
Exception thrown: 'System.Security.SecurityException' in System.dll
Microsoft.WindowsAzure.ServiceRuntime Information: 100 : Role environment . INITIALED RETURNED. HResult=-2147024894
Exception thrown: 'System.Security.SecurityException' in mscorlib.dll
Exception thrown: 'System.Security.SecurityException' in System.dll
Microsoft.WindowsAzure.ServiceRuntime Error: 102 : Role environment . FAILED TO INITIALIZE. hr: -2147024894
Exception thrown: 'System.Security.SecurityException' in mscorlib.dll
Exception thrown: 'System.Security.SecurityException' in System.dll
'iisexpress.exe' (CLR v4.0.30319: Domain 3): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll'
'iisexpress.exe' (CLR v4.0.30319: AppInsightsDomain-19f2071b-dd03-4c5a-9873-ded2cd0d4d19): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\System.Web\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Web.dll'
'iisexpress.exe' (CLR v4.0.30319: AppInsightsDomain-19f2071b-dd03-4c5a-9873-ded2cd0d4d19): Unloaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\assembly\dl3\ff9920db\00b200f9_37f7d201\Microsoft.AI.WindowsServer.dll'
'iisexpress.exe' (CLR v4.0.30319: AppInsightsDomain-19f2071b-dd03-4c5a-9873-ded2cd0d4d19): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\Microsoft.WindowsAzure.ServiceRuntime\v4.0_2.7.0.0__31bf3856ad364e35\Microsoft.WindowsAzure.ServiceRuntime.dll'
'iisexpress.exe' (CLR v4.0.30319: AppInsightsDomain-19f2071b-dd03-4c5a-9873-ded2cd0d4d19): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Xml.Linq\v4.0_4.0.0.0__b77a5c561934e089\System.Xml.Linq.dll'
'iisexpress.exe' (CLR v4.0.30319: AppInsightsDomain-19f2071b-dd03-4c5a-9873-ded2cd0d4d19): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll'
'iisexpress.exe' (CLR v4.0.30319: AppInsightsDomain-19f2071b-dd03-4c5a-9873-ded2cd0d4d19): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Core\v4.0_4.0.0.0__b77a5c561934e089\System.Core.dll'
'iisexpress.exe' (CLR v4.0.30319: AppInsightsDomain-19f2071b-dd03-4c5a-9873-ded2cd0d4d19): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Configuration\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Configuration.dll'
'iisexpress.exe' (CLR v4.0.30319: AppInsightsDomain-19f2071b-dd03-4c5a-9873-ded2cd0d4d19): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Xml\v4.0_4.0.0.0__b77a5c561934e089\System.Xml.dll'
'iisexpress.exe' (CLR v4.0.30319: AppInsightsDomain-19f2071b-dd03-4c5a-9873-ded2cd0d4d19): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\msshrtmi\v4.0_2.7.0.0__31bf3856ad364e35\msshrtmi.dll'
The thread 0x2be8 has exited with code 0 (0x0).
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'Anonymously Hosted DynamicMethods Assembly'. 
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Net.Http\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Net.Http.dll'. Cannot find or open the PDB file.
Application Insights Telemetry (unconfigured): {"name":"Microsoft.ApplicationInsights.Message","time":"2018-03-09T08:09:17.2207578Z","tags":{"ai.location.ip":"127.0.0.1","ai.operation.name":"GET /","ai.cloud.roleInstance":"Lenovo","ai.operation.parentId":"|5E2B5RM0bFM=.fff1e93d_","ai.internal.sdkVersion":"dotnet:2.4.0-32153","ai.operation.id":"5E2B5RM0bFM=","ai.operation.syntheticSource":"SDKTelemetry"},"data":{"baseType":"MessageData","baseData":{"ver":2,"message":"AI: Diagnostic message: Performance counters are unavailable when the application is running under IIS Express. Use EnableIISExpressPerformanceCounters element with a value of 'true' within the Performance Collector Module element to override this behavior."}}}
The parameters to the Event method do not match the parameters to the WriteEvent method. This may cause the event to be displayed incorrectly.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\SMDiagnostics\v4.0_4.0.0.0__b77a5c561934e089\SMDiagnostics.dll'. Cannot find or open the PDB file.
Exception thrown: 'System.InvalidOperationException' in mscorlib.dll
Exception thrown: 'System.Web.HttpException' in System.Web.dll
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.ServiceModel.Internals\v4.0_4.0.0.0__31bf3856ad364e35\System.ServiceModel.Internals.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Web.Mobile\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Web.Mobile.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'A_741f820d_0e4d_4d46_8ad9_72cb07aebfe4'. 
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Web.Entity\v4.0_4.0.0.0__b77a5c561934e089\System.Web.Entity.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Design\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Design.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: DefaultDomain): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Design\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Design.dll'. Cannot find or open the PDB file.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\App_Web_trqwn4lc.dll'. 
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650565537279957): Loaded 'C:\Users\Geoff\AppData\Local\Temp\Temporary ASP.NET Files\vs\cbea3268\5f21c438\App_Web_h5zdqzzr.dll'. 
Exception thrown: 'System.IO.FileNotFoundException' in mscorlib.dll
'iisexpress.exe' (CLR v4.0.30319: DefaultDomain): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Drawing\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Drawing.dll'. Cannot find or open the PDB file.
Application Insights Telemetry (unconfigured): {"name":"Microsoft.ApplicationInsights.Dev.Request","time":"2018-03-09T08:09:17.4579814Z","tags":{"ai.internal.nodeName":"Lenovo","ai.location.ip":"::1","ai.operation.name":"GET /About","ai.cloud.roleInstance":"Lenovo","ai.internal.sdkVersion":"web:2.4.1-1362","ai.operation.id":"A7nebR2ScgE="},"data":{"baseType":"RequestData","baseData":{"ver":2,"id":"|A7nebR2ScgE=.fff1e93e_","name":"GET /About","duration":"00:00:01.9298161","success":true,"responseCode":"200","url":"http://localhost:55958/About","properties":{"DeveloperMode":"true"}}}}
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
Exception thrown: 'System.Threading.ThreadAbortException' in Rollbar.dll
The thread 0x4e98 has exited with code 0 (0x0).
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
Exception thrown: 'System.Threading.ThreadAbortException' in Rollbar.dll
Exception thrown: 'System.Threading.ThreadAbortException' in Rollbar.dll
The thread 0x4760 has exited with code 0 (0x0).
Exception thrown: 'System.Threading.ThreadAbortException' in Rollbar.dll
Exception thrown: 'System.Threading.ThreadAbortException' in Rollbar.dll
Exception thrown: 'System.Threading.ThreadAbortException' in Rollbar.dll
// repeating lines removed
geoffappleford commented 6 years ago

One more thing, debugging against the debug build of Rollbar, exhibits the same behaviour as removing the try catch block as described above.

Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
Exception thrown: 'System.Threading.ThreadAbortException' in Rollbar.dll
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
Exception thrown: 'System.Threading.ThreadAbortException' in System.Web.dll
Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
The thread 0x4798 has exited with code 0 (0x0).
The thread 0x4860 has exited with code 0 (0x0).
The thread 0x49e4 has exited with code 0 (0x0).
'iisexpress.exe' (CLR v4.0.30319: Domain 2): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll'
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650588526728147): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\System.Web\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Web.dll'
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131650588526728147): Unloaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll'
akornich commented 6 years ago

@geoffappleford, thanks for the experimentation! I am a bit confused about step 4 in your repro steps: after running the app in step 3, you are just removing/adding packages using the package manager while app is running? Also, can you get a bit more info if you select the ThreadAbortException in the VS2017 exception settings view and repro it in debug build?

geoffappleford commented 6 years ago

just removing/adding packages using the package manager while app is running?

Correct. The app is fine after starting with or without debugging (F5 or CTRL F5) but the exception occurs immediately after nuget has finished adding or removing the app. I'm using the Nuget UI for this.

To be clear, I get the same high cpu problem in production after deploying the website. These steps using Nuget just repro it locally in VS.

There are 2 exceptions in that method. Firstly on line 144 Thread.Sleep(this._sleepInterval);

System.Threading.ThreadAbortException
  HResult=0x80131530
  Message=Thread was being aborted.
  Source=mscorlib
  StackTrace:
   at System.Threading.Thread.SleepInternal(Int32 millisecondsTimeout)
   at System.Threading.Thread.Sleep(Int32 millisecondsTimeout)

Followed by one on line 141 ProcessAllQueuesOnce(); (in release mode only)

   System.Threading.ThreadAbortException
  HResult=0x80131530
  Message=Thread was being aborted.
  Source=mscorlib
  StackTrace:
   at System.Threading.Monitor.Enter(Object obj)
   at Rollbar.RollbarQueueController.KeepProcessingAllQueues() in C:\dev\github\Rollbar.NET\Rollbar\RollbarQueueController.cs:line 141

Here's a gif showing the error in debug mode. It's a bit slow initially loading all the symbols...

debug

akornich commented 6 years ago

@geoffappleford, thanks a lot for the help! I think I am getting to the bottom of the issue and will need a few more hours to have more elegant solution to the problem, but meanwhile if you could rebuild the Rollbar.NET from the source while swapping existing implementation (in RollbarQueueController.cs) of the following function with provided one:

    private void KeepProcessingAllQueues()
    {
        while(true)
        {
            try
            {
                lock(this._syncLock)
                {
                    ProcessAllQueuesOnce();
                }

                Thread.Sleep(this._sleepInterval);
            }
            catch (System.Threading.ThreadAbortException tae)
            {
                return;
            }
            catch (System.Exception ex)
            {
                //TODO: do we want to direct the exception 
                //      to some kind of Rollbar notifier maintenance "access token"?
            }
        }
    }

it could unblock you until the better solution is ready and published.

trygvelo commented 6 years ago

@akornich We compiled the code from WideSpectrumComputing fork yesterday and deployed to production, works as intended.

I'm nitpicking now, but the issue text for #102 could be misleading (sure had me for a sec), since many people relate the word "recycle" to AppPool. I prefer the term "restart" for AppDomain to avoid confusion. https://stackoverflow.com/a/5651141/288651. Recycling the AppPool was what allowed us to work around the problem in the first place.

Anyways, thanks for getting this fixed quickly! When is 1.1.0 scheduled for release?

akornich commented 6 years ago

@trygvelo, v1.1.0 was just released about an hour ago. Please, give it a try. Any feedback is very welcome.

jonathanjto commented 6 years ago

It's OK for me, it's working

akornich commented 6 years ago

@jonathanjto, thanks for the feedback!

geoffappleford commented 6 years ago

@akornich - 1.1.0 works perfectly. Thanks for getting this fixed so quickly.

akornich commented 6 years ago

@geoffappleford, thanks!