search

HlidacStatu / Hlidac-Statu

Kompletní zdrojový kód pro web hlidacstatu.cz. Pomozte nám rozvíjet a vylepšovat jeden z největších a nejdůležitějších serverů pro transparentnost státu v ČR.
https://www.hlidacstatu.cz
GNU Affero General Public License v3.0
24 stars 7 forks source link

Memory Issues #295

Open suchoss opened 1 year ago

suchoss commented 1 year ago
image

U Kindexu bych navrhoval uložit data do databáze a zbavit se memory cache. Nepřepočítáváme to tak často a jsou to pevné objekty po letech.

suchoss commented 1 year ago

"Unhadled exception >500 \"OnHTTPErrorMidddleware\" \"/Error/500\"?\"?q=osobaid%3Aandrej-babis\"\n Ex Type: Enyim.Caching.Memcached.MemcachedClientException LogMessage: Empty response received. In Method: System.String ReadResponse(Enyim.Caching.Memcached.PooledSocket) Stack Trace: at Enyim.Caching.Memcached.Protocol.Text.TextSocketHelper.ReadResponse(PooledSocket socket) at Enyim.Caching.Memcached.Protocol.Text.GetHelper.ReadItem(PooledSocket socket) at Enyim.Caching.Memcached.Protocol.Text.GetOperation.ReadResponse(PooledSocket socket) at Enyim.Caching.Memcached.MemcachedNode.ExecuteOperation(IOperation op) at Enyim.Caching.MemcachedClient.PerformTryGet(String key, UInt64& cas, Object& value) at Enyim.Caching.MemcachedClient.TryGet(String key, Object& value) at Devmasters.Cache.Memcached.MemcachedCacheProvider1.Exists(String key) at Devmasters.Cache.BaseCache1.Get() at Devmasters.Cache.ManagerBase3.Get(Key key, TimeSpan itemSpecificExpiration) at Devmasters.Cache.ManagerBase3.Get(Key key) at HlidacStatu.Repositories.Osoby.OsobyMCMByNameId.Get(String key) in P:\CCD\hs.team\Hlidac-Statu\Repositories\Temp\Osoby.cs:line 129 at HlidacStatu.Repositories.Searching.Rules.OsobaId.processQueryPart(Part part) in P:\CCD\hs.team\Hlidac-Statu\Repositories\Searching\Rules\OsobaId.cs:line 36 at HlidacStatu.Repositories.Searching.Rules.RuleBase.Process(Part queryPart) in P:\CCD\hs.team\Hlidac-Statu\Repositories\Searching\Rules\RuleBase.cs:line 43 at HlidacStatu.Repositories.Searching.SimpleQueryCreator.GetSimpleQuery(SplittingQuery sq, IRule[] rules) in P:\CCD\hs.team\Hlidac-Statu\Repositories\Searching\SimpleQueryCreator.cs:line 20 at HlidacStatu.Repositories.Searching.SimpleQueryCreator.GetSimpleQuery[T](String query, IRule[] rules) in P:\CCD\hs.team\Hlidac-Statu\Repositories\Searching\SimpleQueryCreator.cs:line 89 at HlidacStatu.Repositories.SmlouvaRepo.Searching.SimpleSearchAsync(String query, Int32 page, Int32 pageSize, OrderResult order, AggregationContainerDescriptor1 anyAggregation, Nullable1 platnyZaznam, Boolean includeNeplatne, Boolean logError, Boolean fixQuery, Boolean withHighlighting, Boolean exactNumOfResults, CancellationToken cancellationToken) in P:\CCD\hs.team\Hlidac-Statu\Repositories\SmlouvaRepo.Search.cs:line 236 at HlidacStatu.Repositories.ES.QueryGrouped.SmlouvyPerYearAsync(String query, Int32[] interestedInYearsOnly) in P:\CCD\hs.team\Hlidac-Statu\Repositories\Temp\QueryGrouped.cs:line 112 at HlidacStatu.Repositories.Statistics.SmlouvyStatistics.CalculateAsync(String query) in P:\CCD\hs.team\Hlidac-Statu\Repositories\Statistics\SmlouvyStatistics.cs:line 34 at AspNetCoreGeneratedDocument.Views_Home_PartialsanalyzaPage.ExecuteAsync() in P:\CCD\hs.team\Hlidac-Statu\Web\Views\Home\Partials\_analyzaPage.cshtml:line 18 at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderPageCoreAsync(IRazorPage page, ViewContext context) at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderPageAsync(IRazorPage page, ViewContext context, Boolean invokeViewStarts) at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderAsync(ViewContext context) at Microsoft.AspNetCore.Mvc.TagHelpers.PartialTagHelper.RenderPartialViewAsync(TextWriter writer, Object model, IView view) at Microsoft.AspNetCore.Mvc.TagHelpers.PartialTagHelper.ProcessAsync(TagHelperContext context, TagHelperOutput output) at Microsoft.AspNetCore.Razor.Runtime.TagHelpers.TagHelperRunner.gAwaited|0_0(Task task, TagHelperExecutionContext executionContext, Int32 i, Int32 count) at AspNetCoreGeneratedDocument.Views_Home_Analyza.b17_2() in P:\CCD\hs.team\Hlidac-Statu\Web\Views\Home\Analyza.cshtml:line 22 at Microsoft.AspNetCore.Razor.Runtime.TagHelpers.TagHelperExecutionContext.GetChildContentAsync(Boolean useCachedResult, HtmlEncoder encoder) at Microsoft.AspNetCore.Mvc.TagHelpers.CacheTagHelper.ProcessContentAsync(TagHelperOutput output) at Microsoft.AspNetCore.Mvc.TagHelpers.CacheTagHelper.CreateCacheEntry(CacheTagKey cacheKey, TagHelperOutput output) at Microsoft.AspNetCore.Mvc.TagHelpers.CacheTagHelper.ProcessAsync(TagHelperContext context, TagHelperOutput output) at Microsoft.AspNetCore.Razor.Runtime.TagHelpers.TagHelperRunner.g__Awaited|0_0(Task task, TagHelperExecutionContext executionContext, Int32 i, Int32 count) at AspNetCoreGeneratedDocument.Views_Home_Analyza.ExecuteAsync() in P:\CCD\hs.team\Hlidac-Statu\Web\Views\Home\Analyza.cshtml:line 21 at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderPageCoreAsync(IRazorPage page, ViewContext context) at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderPageAsync(IRazorPage page, ViewContext context, Boolean invokeViewStarts) at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderAsync(ViewContext context) at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ViewContext viewContext, String contentType, Nullable1 statusCode) at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ViewContext viewContext, String contentType, Nullable1 statusCode) at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ActionContext actionContext, IView view, ViewDataDictionary viewData, ITempDataDictionary tempData, String contentType, Nullable`1 statusCode) at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewResultExecutor.ExecuteAsync(ActionContext context, ViewResult result) at Microsoft.AspNetCore.Mvc.ViewResult.ExecuteResultAsync(ActionContext context) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.gAwaited|30_0[TFilter,TFilterAsync](ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.gAwaited|28_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.gAwaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.gAwaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope) at Microsoft.AspNetCore.Routing.EndpointMiddleware.gAwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger) at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context) at HlidacStatu.LibCore.MiddleWares.ApiAuthenticationMiddleware.Invoke(HttpContext context) in P:\CCD\hs.team\Hlidac-Statu\LibCore\MiddleWares\ApiAuthenticationMiddleware.cs:line 74 at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.ResponseCaching.ResponseCachingMiddleware.Invoke(HttpContext httpContext) at HlidacStatu.Web.Startup.<>c.<b__8_1>d.MoveNext() in P:\CCD\hs.team\Hlidac-Statu\Web\Startup.cs:line 268 --- End of stack trace from previous location ---

suchoss commented 1 year ago

Častý garbage collecting, který jde z infofactů:

image

Tady mi přijde, že na S3 cache dochází k velké deserializaci (škoda, že nejde načíst hotové objekty z cache) - napadá mě možná optimalizace => přechod z newtonsoft json na dotnet json. Snížil by se nejspíš pressure na GC.

suchoss commented 1 year ago

VZ z produkce - hit je docela velký

image