search

aspnet / Logging

[Archived] Common logging abstractions and a few implementations. Project moved to https://github.com/aspnet/Extensions
Apache License 2.0
507 stars 247 forks source link

Logging blocks thread in asp.net core #943

Closed dharmenbavaria closed 5 years ago

dharmenbavaria commented 5 years ago

We have fully async asp.net core application. Which works very well every time even when there is heavy traffic. We use EF core, Redis for caching etc. We had a situation in our production environment when one of our database was out of reach and at that time we started getting 502 and 503 by the time database was out of reach because of many errors. So we tried to simulate this in our development environment by faking IP address in dev machine for the database in the host file. And we were able to reproduce the same situation we started getting 502.

So i created a dump file and started analyzing it with debug mixed only and found that there were many blocking threads with following stacktrace attached an image1 and also image 2 image1

image2

And some were blocked at image3

image

Which were coming mostly from console logger. And the only logger which we have is below we configure at startup and we really dont do console logging.

loggerFactory.AddAWSProvider(Configuration.GetAWSLoggingConfigSection());

Could someone tell me what's going wrong here?

dharmenbavaria commented 5 years ago

The complete application is fully async but I see all the min number of threads blocked at the exception. Can't serve other users if there are so many exceptions. Don't know if someone has better options here how to set up something which still doesn't block if there are exception coming up from one resource even though everything is fully async.