[QUESTION] How should we debug auto-recovery errors?

[simi2525]

Hi, we are using an in memory L1 and Redis L2 cache with FusionCache. Everything works great, however, we are getting this error in our terminal output when we restart our application. [09:40:10 ERR] FUSION [N=FusionCache I=0361226f78fa44ba9c74b98cfedbb859] (O=0HN747TGSUJ6D K=test/OptionCategory/4): stopped auto-recovery because of an error after 1 processed items Can't find a way to debug it and find its cause. We tried to set the logging level to debug but couldn't find relevant logs for this.

[jodydonetti]

Hi @simi2525 , sorry for the delay but I just came back from (very late) summer vacations 😅

The error you are seeing is generate here (line 367).

This happens if the variable hasStopped has been set to true, which as you can see above on line 359 should log another log message with the exception.

Since you say there are no other log messages about such an error and you talked about restarting your app, my instinct tells me that it could be because of line 352, and as you can see the log level used for the OperationCanceledException is not Error but Debug (and you are probably not logging debug-level messages).

All in all it probably means that:

• you restart your app
• a restart is basically a shutdown + start
• during the shutdown, your app disposes the FusionCache instance (as it should)
• disposing FusionCache, in turn, cancels the auto-recovery processing

So, if when shutting down your app (or restarting it, since as we saw it's the same thing) there are some items in auto-reovery queue (because of some transient errors), you'll get that exception.

I'm going to think if there's a better way to handle this, to avoid surprises for other users like you... maybe just log the OperationCanceledException as Error, since it should be rare and it would not create a lot of background noise anyway?

Hope this helps, let me know!