search

ZiggyCreatures / FusionCache

FusionCache is an easy to use, fast and robust hybrid cache with advanced resiliency features.
MIT License
1.65k stars 87 forks source link

[QUESTION] Out of memory deserializing #182

Closed jon-christensen-ag closed 10 months ago

jon-christensen-ag commented 10 months ago

We have recently added caching to our api using FusionCache and, in production, we are getting the below exception when we try to retrieve our list of Users. We are using the FusionCacheNewtonsoftJsonSerializer with default settings:

.WithSerializer(
    new FusionCacheNewtonsoftJsonSerializer()
);

The data isn't huge (at least I don't think so) at 38,000 rows from the database. Any thoughts on what we can do to avoid these errors? Any serializer settings that would help?

We are on v.0.20 of FusionCache and 13.0.3 of Newtonsoft.

System.OutOfMemoryException:
   at System.String.CreateStringFromEncoding (System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Text.UTF8Encoding.GetString (System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Text.Encoding.GetString (System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Text.UTF8Encoding+UTF8EncodingSealed.GetString (System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at ZiggyCreatures.Caching.Fusion.Serialization.NewtonsoftJson.FusionCacheNewtonsoftJsonSerializer.Deserialize (ZiggyCreatures.FusionCache.Serialization.NewtonsoftJson, Version=0.20.0.0, Culture=neutral, PublicKeyToken=null: /_/src/ZiggyCreatures.FusionCache.Serialization.NewtonsoftJson/FusionCacheNewtonsoftJsonSerializer.cs:35)
   at ZiggyCreatures.Caching.Fusion.Serialization.NewtonsoftJson.FusionCacheNewtonsoftJsonSerializer.DeserializeAsync (ZiggyCreatures.FusionCache.Serialization.NewtonsoftJson, Version=0.20.0.0, Culture=neutral, PublicKeyToken=null: /_/src/ZiggyCreatures.FusionCache.Serialization.NewtonsoftJson/FusionCacheNewtonsoftJsonSerializer.cs:47)
   at ZiggyCreatures.Caching.Fusion.Internals.Distributed.DistributedCacheAccessor+<TryGetEntryAsync>d__16`1.MoveNext (ZiggyCreatures.FusionCache, Version=0.20.0.0, Culture=neutral, PublicKeyToken=null: /_/src/ZiggyCreatures.FusionCache/Internals/Distributed/DistributedCacheAccessor_Async.cs:182)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Threading.Tasks.ValueTask`1.get_Result (System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.ConfiguredValueTaskAwaitable`1+ConfiguredValueTaskAwaiter.GetResult (System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at ZiggyCreatures.Caching.Fusion.FusionCache+<GetOrSetEntryInternalAsync>d__45`1.MoveNext (ZiggyCreatures.FusionCache, Version=0.20.0.0, Culture=neutral, PublicKeyToken=null: /_/src/ZiggyCreatures.FusionCache/FusionCache_Async.cs:251)
jodydonetti commented 10 months ago

Hi @jon-christensen-ag and thanks for using FusionCache!

Without more information I can't really know, but since the problem is during deserialization and FusionCache does not provide a serializer but only act as a bridge towards existing ones, and the one you are using seems to be Newtonsoft Json (a.k.a. Json.NET), the problem is most definitely there. Usually in this case it's a circular reference or a wrong configuration with the serializer.

What I would suggest is to try to serialize and deserialize back the same data that you are now caching, using directly Newtonsoft Json, and see what happens: since FusionCache just calls that it's very much probable that you will get the same error.

Hope this helps, let me know if it did work.

jodydonetti commented 10 months ago

To be clear, this is the code that acts as the bridge (a.k.a. adapter) between FusionCache and Newtonsoft Json https://github.com/ZiggyCreatures/FusionCache/blob/main/src/ZiggyCreatures.FusionCache.Serialization.NewtonsoftJson/FusionCacheNewtonsoftJsonSerializer.cs .

As you can see it just forwards the call, that's all.

jodydonetti commented 10 months ago

Closing this, as no answer.

Will reopen if needed.