OptionsCache fails to recover from initialization failure

[RaffaelHold]

Description

When passing a factory method - that might throw a runtime exception - as a parameter to .GetOrAdd() on an instance of OptionsCache the cache will never recover if an exception gets thrown in the factory method.

This happens because the underlying Lazy will never try to reinitialize its value even if the error condition goes away.

For example Microsoft.Identity.Web does make an remote HTTP call during initialization of JwtBearerOptions. If theres a connection outage, when the factory method is executed, every subsequent cache access will throw an exception.
The only fix is to manually clear the cache or restart the application.

Sample:

using System;
using Microsoft.Extensions.Options; // Microsoft.Extensions.Options @ 3.1.7

namespace ConsoleApp
{
    public static class Program
    {
        static void Main()
        {
            var optionsCache = new OptionsCache<Options>();
            try
            {
                var expectedToFail = optionsCache.GetOrAdd("options", () => new Options());
            }
            catch { }

            var shouldBeSucceeding = optionsCache.GetOrAdd("options", () => new Options());
        }

        private class Options
        {
            private static bool _throws = true;
            public Options()
            {
                if (_throws)
                {
                    _throws = false;
                    throw new Exception();
                }
            }
        }
    }
}

Possible Solutions

a) change the Lazy's LazyThreadSafetyMode to PublicationOnly to allow multiple evaluations until one succeeds (this might have unintended performance consequences)
b) reset the cache entry when an exception is thrown during initialization of the Lazy

Configuration

• Which version of .NET is the code running on?
  3.1.7
• What OS and version, and what distro if applicable?
  Windows 10 2004 Build 19041.450
• What is the architecture (x64, x86, ARM, ARM64)?
  x64
• Do you know whether it is specific to that configuration?
  It is not specific to configuration

[Dotnet-GitSync-Bot]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ghost]

Tagging subscribers to this area: @maryamariyan See info in area-owners.md if you want to be subscribed.

[ViktorHofer]

@RaffaelHold are you sending the HTTP request during the construction of your Options instance?

[RaffaelHold]

@ViktorHofer I found this issue in Microsoft.Identity.Web where indeed an HTTP request is sent during construction of the Options instance. But any runtime exception will lead to this behavior

Here are the relevant lines in Microsoft.Identity.Web Microsoft.Identity.Web/WebApiExtensions/WebApiAuthenticationBuilderExtensions.cs which leads to a HTTP call being made in AadIssuerValidator.GetIssuerValidator

I hope this gives you a better idea what's happening.

I'd also be happy to give the implementation of a fix a shot if I get some pointers on how we want to resolve this issue.

[davidfowl]

Don't make http requests that block during construction of objects. Do it before or after. This could even result in deadlocks if not careful.

Even if the lazy doesn't re-evaluate, this looks like a bad idea and it's weird that options would be re-initialized and retired. Constructions of objects isn't something that should be unreliable and retried (or bake the retries into the code itself)