alastairtree / LazyCache

An easy to use thread safe in-memory caching service with a simple developer friendly API for c#
https://nuget.org/packages/LazyCache
MIT License
1.72k stars 159 forks source link

Maybe a problem... #14

Closed EssyTech closed 6 years ago

EssyTech commented 7 years ago

This library was easy to implement into my repository of my web api. However, when I moved it to production I noticed that my cpu would gradually rise and never drop down, all the way up to 70-75 percent and stay there. As soon as I cleared the cache on the site, the cpu would go back to 0 - 5 percent and slowly begin to creep up. I can manage it using the recycler in IIS but I need to take a deeper look under the hood to see what the creep is. However, if you have a suggestion or idea of what is happening, that would be great.

alastairtree commented 7 years ago

Are you able to give any more information about what you are caching, how often, duration etc. Usually high CPU suggests an issue in garbage collection but could be so many things. Strange that it decreases on clear. Any kind of profiling you are able to do would help diagnose the issue.

EssyTech commented 7 years ago

Well, I try to cache a lot of my high data requests. This includes authentication user data, project data, notification data. I check the amount of items in the cache and at peak times its between 3000 - 4000 items, memory usage is between 500mb - 1gig, never higher. I kept the default expiration of 20 minutes for most the data. For static data that rarely changes I cache for 24 hours. The cpu does this: 0-5 percent then a little bit later 5-10 percent then 10-15 percent and creeps up like that, never goes down. One thing I noticed is that I can remove all items in the cache using a secure api call and the cpu never goes down until I recycle the application pool. Weird. However, it didnt start happening until I implemented this caching solution. I will try to profile it but I have many other small issues to work out in other parts of the application. Any ideas?

alastairtree commented 7 years ago

Are you using the cache item policy removal callbacks? Have been trying to do some profiling and suspect there may be a memory leak because the code that unwraps the lazy for the callbacks is holding onto references so that they can never be disposed. Just speculating but this might lead to garbage collection consuming CPU. When you are experiencing issues what percentage of memory is free on that box?

EssyTech commented 7 years ago

I have not tried using the removal callbacks. This would be used to investigate? I still haven't had a chance to profile on my end. The server has 32 gigs of ram and over 70 percent is free at most times.

alastairtree commented 7 years ago

If you are not using the callbacks then i don't know. When I have done some profiling the callback policies were the only thing not getting picked up by the GC. Still would be useful to know if memory pressure was triggering the high CPU as a result of GC activity

alastairtree commented 6 years ago

I have not been able to reproduce this issue, and I do not know of anyone else suffering from it either. I am also working on a netstandard version of the app for version 2.0 that is a significant rewrite and so any issue would need recreating/fixing within the context that version anyway. I am going to close this issue for now but please shout if you still think this is an ongoing issue.

Eugene-Lytvynov commented 4 months ago

We have same issue, in some period of time.

var xsltTransformer = _cache.GetOrAdd(
    key,
    () =>
    {
        var settingsCompledTransofrm = new XsltSettings(XML_XSLT_FUNCTION, XML_XSLT_SCRIPTING);

        XslCompiledTransform transformer = new XslCompiledTransform(XML_XSLT_DEBUGING);

        if (!File.Exists(fullFileNameXSL)) { throw new Exception("File not exist: " + fullFileNameXSL); }

        transformer.Load(
            fullFileNameXSL,
            settingsCompledTransofrm,
            new XmlUrlResolver()
        );

        //Debug.WriteLine("!!! Factory transform for: " + fullFileNameXSL);
        return transformer;

    },
    TimeSpan.FromMinutes(15)
);